Managing data in a storage system

ABSTRACT

According to an aspect of the presently disclosed subject matter, there is provided a system for managing data in a storage system, the system including a storage layer which provides storage resource, and a snapshot layer that includes: a volume-version data structure, a chunk-version data structure and a IO handler.

FIELD OF THE INVENTION

The present invention is in the field of storage systems and relates tomanaging data in a storage system that implements snapshots.

SUMMARY

Many of the functional components of the presently disclosed subjectmatter can be implemented in various forms, for example, as hardwarecircuits comprising custom VLSI circuits or gate arrays, or the like, asprogrammable hardware devices such as FPGAs or the like, or as asoftware program code stored on an intangible computer readable mediumand executable by various processors, and any combination thereof. Aspecific component of the presently disclosed subject matter can beformed by one particular segment of software code, or by a plurality ofsegments, which can be joined together and collectively act or behaveaccording to the presently disclosed limitations attributed to therespective component. For example, the component can be distributed overseveral code segments such as objects, procedures, and functions, andcan originate from several programs or program files which operate inconjunction to provide the presently disclosed component.

In a similar manner, a presently disclosed component(s) can be embodiedin operational data or operational data can be used by a presentlydisclosed component(s). By way of example, such operational data can bestored on tangible computer readable medium. The operational data can bea single data set, or it can be an aggregation of data stored atdifferent locations, on different network nodes or on different storagedevices.

According to an aspect of the presently disclosed subject matter, thereis provided a method of managing data in a storage system. According toexamples of the presently disclosed subject matter, the method ofmanaging data in a storage system can include: maintaining avolume-version data structure including LU volume entries for a LUvolume and for snapshot volumes associated with the LU volume, whereinfor each volume a respective volume ID is kept, and for each snapshot atimestamp for the snapshot operation and an indication of a connectionbetween a respective target volume and a source volume are kept;maintaining a chunk-version data structure that is uniquely associatedwith a volume ID and a logical chunk address combination; including inthe chunk-version data structure one or more chunk-version dataelements, each chunk version data element including a timestamp at acorresponding write to the respective logical chunk address in therespective volume and a pointer to storage resources associated with thechunk-version data element.

According to an aspect of the presently disclosed subject matter, thereis provided a storage system. According to examples of the presentlydisclosed subject matter, the storage layer can include a storage layerwhich provides storage resource, and a snapshot layer that includes: avolume-version data structure, a chunk-version data structure and a IOhandler. The volume-version data structure can include LU volume entriesfor an LU volume and for snapshot volumes associated with the LU volume,wherein for each volume a respective volume ID can be kept, and for eachsnapshot a timestamp for the snapshot operation and an indication of aconnection between a respective target volume and a source volume can bekept. The chunk-version data structure can be uniquely associated with avolume ID and a logical chunk address combination. The IO handler can beconfigured to include in the chunk-version data structure one or morechunk-version data elements, each chunk version data element including atimestamp at a corresponding write to the respective logical chunkaddress in the respective volume and a pointer to storage resourcesassociated with the chunk-version data element.

According to yet another aspect of the presently disclosed subjectmatter, there is provided a program storage device readable by machine,tangibly embodying a program of instructions executable by the machineto perform a method of managing data in a storage system. According toexamples of the presently disclosed subject matter, the program ofinstructions executable by the machine can include instructions: formaintaining a volume-version data structure including LU volume entriesfor an LU volume and for snapshot volumes associated with the LU volume,wherein for each volume a respective volume ID is kept, and for eachsnapshot a timestamp for the snapshot operation and an indication of aconnection between a respective target volume and a source volume arekept; for maintaining a chunk-version data structure that is uniquelyassociated with a volume ID and a logical chunk address combination; andfor including in the chunk-version data structure one or morechunk-version data elements, each chunk version data element including atimestamp at a corresponding write to the respective logical chunkaddress in the respective volume and a pointer to storage resourcesassociated with the chunk-version data element.

According to an aspect of the presently disclosed subject matter, thereis provided a computer program product comprising a computer useablemedium having computer readable program code embodied therein formanaging data in a storage system. According to examples of thepresently disclosed subject matter, the computer program product caninclude: computer readable program code for causing the computer tomaintain a volume-version data structure including LU volume entries foran LU volume and for snapshot volumes associated with the LU volume,wherein for each volume a respective volume ID is kept, and for eachsnapshot a timestamp for the snapshot operation and an indication of aconnection between a respective target volume and a source volume arekept; computer readable program code for causing the computer tomaintain a chunk-version data structure that is uniquely associated witha volume ID and a logical chunk address combination; and computerreadable program code for causing the computer to include in thechunk-version data structure one or more chunk-version data elements,each chunk version data element including a timestamp at a correspondingwrite to the respective logical chunk address in the respective volumeand a pointer to storage resources associated with the chunk-versiondata element.

According to still a further aspect of the presently disclosed subjectmatter, there is provided a method of handling writes in a storagesystem. According to examples of the presently disclosed subject matter,the method of handling writes in a storage system can include inresponse to receiving a write command determining with which snapshotlayer chunk the write command is associated, and whether there is anexisting chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated; in casethe write command is associated with an existing chunk-version datastructure, reading a timestamp from a last chunk-version data element inthe existing chunk-version data structure, and if there is a snapshotoperation whose source is a volume referenced in the write command andthis snapshot operation is associated with a timestamp that is laterthan a timestamp of the last chunk-version data element, maintaining thelast chunk-version data element and adding to the existing chunk-versiondata structure a new chunk version data element for the write command.

According to another aspect of the presently disclosed subject matter,there is provided a storage system. According to examples of thepresently disclosed subject matter, the storage layer can include astorage layer which provides storage resource, and a snapshot layer thatincludes: a volume-version data structure, a chunk-version datastructure and a IO handler. The IO handler can be responsive toreceiving a write command in the snapshot layer for determining withwhich snapshot layer chunk the write command is associated, and whetherthere is an existing chunk version data structure which is associatedwith the snapshot layer chunk with which the write command isassociated. In case the write command is associated with an existingchunk-version data structure, the IO handler can be configured to read atimestamp from a last chunk-version data element in the existingchunk-version data structure. If there is a snapshot operation whosesource is a volume referenced in the write command and this snapshotoperation is associated with a timestamp that is later than a timestampof the last chunk-version data element, the IO handler can be configuredto maintain the last chunk-version data element and to add to theexisting chunk-version data structure a new chunk version data elementfor the write command.

According to yet another aspect of the presently disclosed subjectmatter, there is provided a program storage device readable by machine,tangibly embodying a program of instructions executable by the machineto perform a method of handling writes in a storage system. According toexamples of the presently disclosed subject matter, the program ofinstructions executable by the machine can include instructions: fordetermining, in response to receiving a write command, with whichsnapshot layer chunk the write command is associated, and whether thereis an existing chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated; forreading, in case the write command is associated with an existingchunk-version data structure, a timestamp from a last chunk-version dataelement in the existing chunk-version data structure, and formaintaining the last chunk-version data element and adding to theexisting chunk-version data structure a new chunk version data elementfor the write command, if there is a snapshot operation whose source isa volume referenced in the write command and this snapshot operation isassociated with a timestamp that is later than a timestamp of the lastchunk-version data element.

According to an aspect of the presently disclosed subject matter, thereis provided a computer program product comprising a computer useablemedium having computer readable program code embodied therein forhandling writes in a storage system. According to examples of thepresently disclosed subject matter, the computer program product caninclude: computer readable program code that is responsive to receivinga write command in the storage system for causing the computer todetermine with which snapshot layer chunk the write command isassociated, and to determine whether there is an existing chunk versiondata structure which is associated with the snapshot layer chunk withwhich the write command is associated; computer readable program codefor causing the computer to determine whether the write command isassociated with an existing chunk-version data structure, and in casethe write command is associated with an existing chunk-version datastructure, the computer readable program code is configured to cause atimestamp from a last chunk-version data element in the existingchunk-version data structure to be read, and if there is a snapshotoperation whose source is a volume referenced in the write command andthis snapshot operation is associated with a timestamp that is laterthan a timestamp of the last chunk-version data element, the computerreadable program code is configured to cause the last chunk-version dataelement to maintained and a new chunk version data element for the writecommand to be added to the existing chunk-version data structure.

According to yet another aspect of the presently disclosed subjectmatter, there is provided a method of handling reading in a storagesystem. According to examples of the presently disclosed subject matter,the method of handling reads in a storage system can include: (a) inresponse to receiving a read command at the storage system determiningwith which volume and with which snapshot layer chunk the read commandis associated, initializing a read pointer with respective volume ID andlogical chunk address, and initializing a time-marker according to atimestamp at receipt of the read command; (b) determining whether thereis a chunk-version data structure that is associated with a current readpointer; (c1) if the condition in operation (b) is not met, determiningwhether there is a snapshot operation where the volume that isreferenced in the write pointer is a target volume and whether thesnapshot operation is associated with a timestamp that is not greaterthan the current time-marker; and (d1) if the conditions in operation(c) are met updating the read pointer with a volume ID of a sourcevolume of the snapshot operation, updating the time-marker with thetimestamp for the snapshot operation—1, and returning to operation (b).

According to another aspect of the presently disclosed subject matter,there is provided a storage system. According to examples of thepresently disclosed subject matter, the storage layer can include astorage layer which provides storage resource, and a snapshot layer thatincludes: a volume-version data structure, a chunk-version datastructure and a IO handler. The IO handler can be responsive toreceiving a write command for: (a) determining with which volume andwith which snapshot layer chunk the read command is associated,initializing a read pointer with respective volume ID and logical chunkaddress, and initializing a time-marker according to a timestamp of theread command; (b) determining whether there is a chunk-version datastructure that is associated with a current read pointer; (c1) if thecondition in operation (b) is not met, determining whether there is asnapshot operation where the volume that is referenced in the writepointer is a target volume and whether the snapshot operation isassociated with a timestamp that is not greater than the currenttime-marker; and (d1) if the conditions in operation (c) are met the IOunder is configured for: updating the read pointer with a volume ID of asource volume of the snapshot operation, updating the time-marker withthe timestamp for the snapshot operation—1, and returning to operation(b).

According to yet another aspect of the presently disclosed subjectmatter, there is provided a program storage device readable by machine,tangibly embodying a program of instructions executable by the machineto perform a method of handling reads in a storage system. According toexamples of the presently disclosed subject matter, the program ofinstructions executable by the machine can include instructions: for (a)determining, in response to receiving a read command at the storagesystem, with which volume and with which snapshot layer chunk the readcommand is associated, initializing a read pointer with respectivevolume ID and logical chunk address, and initializing a time-markeraccording to a timestamp of the read command; for (b) determiningwhether there is a chunk-version data structure that is associated witha current read pointer; for (c1) determining, if the condition inoperation (b) is not met, whether there is a snapshot operation wherethe volume that is referenced in the write pointer is a target volumeand whether the snapshot operation is associated with a timestamp thatis not greater than the current time-marker; and for (d1) updating, ifthe conditions in operation (c) are met, the read pointer with a volumeID of a source volume of the snapshot operation, updating thetime-marker with the timestamp for the snapshot operation—1, andreturning to operation (b).

BRIEF DESCRIPTION OF THE DRAWINGS

In order to understand the invention and to see how it may be carriedout in practice, a preferred embodiment will now be described, by way ofnon-limiting example only, with reference to the accompanying drawings,in which:

FIG. 1 is a block diagram illustration of a storage system whichincludes a snapshot layer for supporting and managing snapshot volumesand data versioning, according to examples of the presently disclosedsubject matter;

FIG. 2 is a schematic illustration of one example of a possibleimplementation of the volume-version data structure, according toexamples of the presently disclosed subject matter;

FIG. 3 is a flowchart illustration of method of managing a storagesystem, according to examples of the presently disclosed subject matter;

FIG. 4 is a graphical illustration of a chunk-version data structure,and the chunk-version data elements contained therein at two differenttime instants, according to examples of the presently disclosed subjectmatter;

FIG. 5 is a block diagram illustration of a storage system whichincludes a snapshot layer for supporting and managing snapshot volumesand data versioning;

FIG. 6 is a flowchart illustration of a method of handling writes in astorage system, according to examples of the presently disclosed subjectmatter;

FIG. 7 is a graphical illustration of a chunk-version data structure,and the chunk-version data elements contained therein, at different timeinstants, according to examples of the presently disclosed subjectmatter;

FIG. 8 is a flowchart illustration of a method of handling reads in astorage system, according to examples of the presently disclosed subjectmatter; and

FIG. 9 is a graphical illustration of one aspect of an example of ascenario that involves a series of operations that are carried outaccording to examples of the presently disclosed subject matter.

It will be appreciated that for simplicity and clarity of illustration,elements shown in the figures have not necessarily been drawn to scale.For example, the dimensions of some of the elements may be exaggeratedrelative to other elements for clarity. Further, where consideredappropriate, reference numerals may be repeated among the figures toindicate corresponding or analogous elements.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the presentlydisclosed subject matter. However, it will be understood by thoseskilled in the art that the presently disclosed subject matter may bepracticed without some of these specific details. In other instances,well-known methods, procedures and components have not been described indetail so as not to obscure the presently disclosed subject matter.

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specification variousfunctional terms refer to the action and/or processes of a computer orcomputing device, or similar electronic computing device, thatmanipulate and/or transform data represented as physical, such aselectronic, quantities within the computing device's registers and/ormemories into other data similarly represented as physical quantitieswithin the computing device's memories, registers or other such tangibleinformation storage, transmission or display devices.

It is appreciated that, unless specifically stated otherwise, certainfeatures of the presently disclosed subject matter, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the presently disclosed subject matter, which are, forbrevity, described in the context of a single embodiment, may also beprovided separately or in any suitable sub-combination.

As used herein, the terms “example”, “for example,” “such as”, “forinstance” and variants thereof describe non-limiting embodiments of thepresently disclosed subject matter. Reference in the specification to“one case”, “some cases”, “other cases” or variants thereof means that aparticular feature, structure or characteristic described in connectionwith the embodiment(s) is included in at least one embodiment of thepresently disclosed subject matter. Thus the appearance of the phrase“one case”, “some cases”, “other cases” or variants thereof does notnecessarily refer to the same embodiment(s).

The operations in accordance with the teachings herein may be performedby a computer specially constructed for the desired purposes or by ageneral purpose computer specially configured for the desired purpose bya computer program stored in a non-transitory computer readable storagemedium.

Embodiments of the presently disclosed subject matter are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the presently disclosed subject matter asdescribed herein.

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specificationdiscussions utilizing terms such as “processing”, “obtaining”,“utilizing”, “determining”, “generating”, “setting”, “configuring”,“selecting”, “searching”, “receiving”, “storing” or the like, includeactions and/or processes of a computer that manipulate and/or transformdata into other data, said data represented as physical quantities, e.g.such as electronic quantities, and/or said data representing thephysical objects. The terms “computer”, “processor”, and “controller”should be expansively construed to cover any kind of electronic devicewith data processing capabilities/

According to an aspect the presently disclosed subject matter, there isdisclosed a method of managing data in a storage system. According toexamples of the presently disclosed subject matter, the method caninclude maintaining a volume-version data structure including LU volumeentries for an LU volume and for snapshot volumes associated with the LUvolume, wherein for each volume a respective volume ID is kept, and foreach snapshot a timestamp for the snapshot operation and an indicationof a connection between a respective target volume and a source volumeare kept; maintaining a chunk-version data structure that is uniquelyassociated with a specific snapshot-layer chunk; and including in thechunk-version data structure one or more chunk-version data elements,each chunk version data element including a timestamp at a correspondingwrite to the respective snapshot layer chunk and a pointer to storageresources associated with the chunk-version data element. It would beappreciated that according to examples of the presently disclosedsubject matter, the snapshot operations can be implemented with respectto consistency groups. Each consistency group can include two or morevolumes (with zero or more LU volume and zero or more snapshot volumes)and is regarded as a single entity for the purposes of taking snapshots.A snapshot of a consistency group, in effect, produces a snapshot ofeach volume (LU volume and/or snapshot volume) that is a member of theconsistency group. It would be appreciated that a snapshot createcommand that is directed to a consistency group can be carried out as aplurality of snapshot create commands one with respect to each one ofthe plurality of (volume) members of the consistency group. It should benoted that throughout the description and in the claims, the term“volume” (including LU volume and snapshot volume) also means aconsistency group, and that examples of the presently disclosed subjectmatter which relate to a volume (including LU volume and snapshotvolume) also relate to a consistency group.

According to examples of the presently disclosed subject matter, eachsnapshot layer chunk is associated with a specific volume ID and logicalchunk address combination, and the key of the chunk-version datastructure is the volume ID and logical chunk address combination withwhich the respective snapshot layer chunk is associated.

According to further examples of the presently disclosed subject matter,the storage resources in the chunk-version data element are the storageresources that were allocated for storing the snapshot layer chunk data.

According to an aspect the presently disclosed subject matter, there isdisclosed a storage system. According to examples of the presentlydisclosed subject matter, the storage system according to examples ofthe presently disclosed subject matter can include a central controller,a snapshot layer and a storage layer. Further by way of example, thesnapshot layer can include a volume-version data structure and achunk-version data structure. The volume-version data structure caninclude LU volume entries for an LU volume and for snapshot volumesassociated with the LU volume. For each volume, the volume-version datastructure can hold a respective volume ID, and for each snapshot, thevolume-version data structure can hold a timestamp for the snapshotoperation and an indication of a connection between a respective targetvolume and a source volume. The chunk version data structure is uniquelyassociated with a volume ID and a logical chunk address combination, andhold one or more chunk-version data elements. Each chunk-version dataelement includes a timestamp at a corresponding write to the respectivelogical chunk address in the respective volume and a pointer to storageresources associated with the chunk-version data element

Throughout the description and the claims, reference is made to the term“logical unit volume” or “LU volume” in abbreviation. The term LU volume(or logical unit volume) is known in the art of digital storage, and thefollowing definition is provided as a non-limiting example only forconvenience purposes. Accordingly, the interpretation of the term LUvolume in the claims, unless stated otherwise, is not limited to thedefinitions below and the term LU volume should be given its broadestreasonable interpretation. The term LU volume as used herein relates toa logical unit created by a volume create command or an equivalentcommand. For example, in the SCSI storage protocol, an LU volume iscreated by such a volume create command.

Throughout the description and the claims, reference is made to the term“snapshot volume”. The term snapshot volume is known in the art ofdigital storage, and the following definition is provided as anon-limiting example only for convenience purposes. Accordingly, theinterpretation of the term snapshot volume in the claims, unless statedotherwise, is not limited to the definitions below and the term snapshotvolume should be given its broadest reasonable interpretation. The termsnapshot volume as used herein relates to a logical unit created by asnapshot create command or by an equivalent command. For example, in theSCSI storage protocol, a snapshot volume is created by such a snapshotcreate command. In accordance with examples of the presently disclosedsubject matter, in response to receiving a snapshot create commandreferencing a certain source volume (the source volume can be a LUvolume or another snapshot volume) and a target volume, the followingset of operations can be implemented: I/Os to the source volume can besuspended; a timestamp or counter can be incremented; data structureswhich are related to the snapshot operation can be updated; and the lockon the source volume can be released allowing the I/Os to the sourcevolume to resume. It would be appreciated that according to examples ofthe presently disclosed subject matter, the counter can be (but is notnecessarily) a logical counter that is incremented by a whole number foreach operation at the volume level (e.g., snapshot create, volumecreate, volume revert/restore, etc.). The term timestamp relates to areading from the counter, and represent the state of the counter at agiven instant. In this regard, in the case of a logical counter, atimestamp at a certain write command represents the state of the counterfollowing the most recent volume level operation, and at a given counterstate several writes can be processed, and each of which would beassociated with the same timestamp.

For convenience, in the description of examples of the presentlydisclosed subject matter, reference is made to a logical counter.However, further examples of the presently disclosed subject matter canbe implemented with other types of timestamps or counters including forexample a timestamp that is generated using an external clock, and insuch an implementation instead of the timestamp increment operation theexternal clock can be read and the relevant time record. Those versed inthe art can readily apply such alternative timestamp implementations tothe teachings provided here.

According to examples of the presently disclosed subject matter, avolume that is replicated by a LU revert command or by a LU restore canalso be regarded herein as a snapshot volume. Further according toexamples of the presently disclosed subject matter, a revert or restoreoperation as it is referred to herein is an operation in which the dataof a certain source volume is copied in a snapshot operation to a targetvolume that is associated with a certain existing (in the same volumeversion data structure) volume ID. The handling of a revert or restoreoperation according to examples of the presently disclosed subjectmatter, is further described below. For convenience, except wherereference is particular made to a revert or restore operation, anyreference that is made to a snapshot operation is applicable also to arevert or restore operation, mutatis mutandis.

A snapshot operation involves replicating a certain source volume at acertain point in time to provide a target snapshot volume. Usually whenthe snapshot volume is initiated, a set of references (pointers)pointing to storage resources from the storage layer that were allocatedto the source volume are used, and when new data is written to thetarget volume new storage resources are allocated and the references areupdated to point to the new resources. This type of snapshotimplementation is sometimes referred to as “copy on write” or COW inabbreviation. A “split-mirror” snapshot implementation references allthe data on a set of mirrored drives. Every time a snapshot is created,the entire volume is copied, not only the new or updated data. Examplesof the presently disclosed subject matter are applicant to storagesystem which implement COW snapshots.

Throughout the description and the claims, reference is made to the term“snapshot layer”. The term snapshot layer is known in the art of digitalstorage, and the following definition is provided as a non-limitingexample only for convenience purposes. Accordingly, the interpretationof the term snapshot layer in the claims, unless stated otherwise, isnot limited to the definitions below and the term snapshot layer shouldbe given its broadest reasonable interpretation. The term snapshot layeras used herein, relates to a logical layer in a storage system which isused to manage snapshot operations including mapping of differentversions of data in the storage system's volumes. The snapshot layer canbe capable of managing volume creation, data versioning and resourceallocation, including storage resource allocations to snapshot layerchunks and mapping of external addresses to snapshot layer chunks andvice-versa. It would be appreciated that a snapshot layer can bedesigned to provide high-availability and fault tolerance.

Throughout the description and the claims, reference is made to the term“storage layer”. The term storage layer is known in the art of digitalstorage, and the following definition is provided as a non-limitingexample only for convenience purposes. Accordingly, the interpretationof the term storage layer in the claims, unless stated otherwise, is notlimited to the definitions below and the term storage layer should begiven its broadest reasonable interpretation. The term storage layer asused herein relates to a storage system layer which allocates storageresources, for example, a highly available disk array. The storage layercan be a virtual layer that is implemented over a plurality of units,such as hard drive disks, flash drives etc. The storage layer canprovide a single address space for storing the data set of the storagesystem. From the perspective of the snapshot layer the storage layerprovides a set of storage resource addresses, where the data can bewritten or from which data can be read. The snapshot layer can hold amap or other data structures in which the storage resources addressesallocated by the storage layer for LU volume chunks and for snapshotvolume chunks are recorded.

The term “snapshot layer chunk”, as used herein relates to the basicallocation unit that is managed by the snapshot layer. For each snapshotlayer chunk, a fixed number of consecutive storage layer blocks can beallocated. According to examples of the presently disclosed subjectmatter, each chunk in the snapshot layer can be associated with aspecific logical chunk address, or LCA in abbreviation. Furtheraccording to examples of the presently disclosed subject matter, aunique identifier of a snapshot layer chunk can be a combination of anLCA and a volume ID, and for each snapshot layer chunk different storagelayer resources can be allocated. For example, <LU1, LCA 1> can denote afirst snapshot layer chunk which is separate and distinct from a secondsnapshot layer chunk <LU2, LCA1>.

Yet further according to examples of the presently disclosed subjectmatter, a chunk and the respective storage layer resources can beallocated when the storage layer resources are required by the snapshotlayer, as will be described below.

The term “logical block address” or LBA in abbreviation as used herein,relates to the basic reference unit which is used by the storage systemand external hosts. According to examples of the presently disclosedsubject matter, the snapshot layer is configured to translate LBAs in agiven volume to a respective LCA in the volume. For example, the LBAscan be numerical and range from 1-n, and the corresponding LCA can bederived by rounding down of the LBA address to the nearest LCA,according to the number of LBAs with which each LCA is associated. Insome examples of the presently disclosed subject matter, a partly or anentirely overlapping LBA range (e.g., 1-n) can be used with multiple,different volumes (e.g., all the volumes) in the storage system, andtherefore in order to reference a specific logical block or snapshotlayer chunk, a reference to the volume with which the logical block orsnapshot layer chunk is associated may be required. Accordingly, in someexamples of the presently disclosed subject matter, a logical block anda snapshot layer chunk are referenced by a combination of a respectivevolume ID and LBA or LCA. The combinations <volume ID, LBA> and <volumeID, LCA> are unique within the storage system. It would be appreciatedthat this sort of addressing scheme can be made compatible with someexisting storage protocols, such as SCSI for example.

It should be appreciated that the above addressing scheme is merely onepossible implementation of an addressing scheme and is used herein byway of example only, and in further examples of the presently disclosedsubject matter, different addressing schemes may be used. For example,in a different addressing scheme, each volume in the storage system canbe associated with a different range of LBAs and respective LCAs, and insuch a case there is no need to reference the volume ID with the LBAs orLCAs.

The storage layer can also implement different addressing schemes andcan manage the storage resources in different ways. For example, thebasic storage layer resource can be a “storage layer block” or SRA inabbreviation, and this can be the granularity at which the storage layeroperates in terms of locking and metadata (including the maintenance andmanagement of validity indications). In some examples of the presentlydisclosed subject matter, the storage layer can implement a furthermanagement layer over the “SRAs” and can allocate storage resources tothe snapshot layer in “segments” where each segment spans a predefinednumber of SRAs, typically consecutive SRAs. For the sake ofillustration, and by way of example only, the storage system can use thefollowing units: 128 kb snapshot layer chunks, 4 kb storage layersegments, 512 bytes LBAs and 512 bytes SRAs.

As mentioned above, and as will be described in greater detail below,according to examples of the presently disclosed subject matter, thestorage system can implement and manage an external addressing scheme(the one that is used between the storage system and external hosts), asnapshot layer addressing scheme and a storage layer addressing scheme,and each of these addressing schemes can have a different granularity.Allocation tables and management modules may be required to allow aseamless operation across the various storage layers and with theexternal hosts. For convenience, the description initially relates to astorage system in which the snapshot layer addressing scheme and thestorage layer addressing scheme have the same granularity. Furtherexamples of the presently disclosed subject matter, some of which aredescribed below, relate to storage systems in which the snapshot layeraddressing scheme and the storage layer addressing scheme have adifferent granularity.

Reference is initially made to FIG. 1 which is a block diagramillustration of a storage system which includes a snapshot layer forsupporting and managing snapshot volumes and data versioning, accordingto examples of the presently disclosed subject matter. According toexamples of the presently disclosed subject matter, the storage system100 can include a central controller 10, a snapshot layer 20 and astorage layer 30. The storage system 100 can also include an interface40, for interfacing the storage system 100 with hosts 50.

According to examples of the presently disclosed subject matter, thesnapshot layer 120 can include a volume-version data structure 122 andchunk-version data structures 124. The volume-version data structure 122can include volume entries for an LU volume and for snapshot volumesassociated with the LU volume.

It would be appreciated that the storage system 100 can allocate aplurality of LU volumes (for example, via a volume create command), andsome or all such volumes can be independent logical storage entities. Itwould be appreciated that some operations in a storage system areexclusively defined with respect to volumes (LU volumes and/or snapshotvolumes). For example, volume create, snapshot create and restore createare operations which are defined with respect to volumes. Furthermore,LU volumes can be associated with snapshot volumes, and independent setsof logical volumes can thus be formed. Further by way of example and forillustration purposes, such independent sets of logical volumes can beeach associated with a distinct and separate volume-version datastructure, which is maintained in the snapshot layer 120. Forsimplicity, many of the examples disclosed herein relate to one suchvolume-version data structure 122. However, examples of the presentlydisclosed subject matter are applicable to, and encompass, a storagesystem and a snapshot layer which has a plurality of volume-version datastructures (for example, two three, . . . , n). It would also beappreciated that the volume-version data can be organized in any otherform or structure as will be known to those versed in the art.

For each volume (LU volume or snapshot volume), the volume-version datastructure 122 can hold a respective volume ID, and for each snapshot,the volume-version data structure 122 can hold a timestamp for thesnapshot operation and an indication of a connection between arespective target volume and a source volume. It would be appreciatedthat a source volume of a given snapshot operation can be a LU volume oranother snapshot volume.

Reference is now additionally made to FIG. 2, which is a schematicillustration of one example of a possible implementation of thevolume-version data structure, according to examples of the presentlydisclosed subject matter. As can be seen in FIG. 2, each volume isassociated with a respective volume ID (e.g., LU1, LU2, . . . , LUn).According to examples of the presently disclosed subject matter, eachone of the volume IDs is unique within the volume-version data structure200, and in case more than one volume-version data structures exist, thevolume ID's can also be unique across the multiple volume-version datastructures. In this regard, it should be appreciated that the addressingscheme that is used for identifying and marking the volumes is notlimited to what is shown in FIG. 2 and described herein, and any othersuitable addressing or identification schemes can be used in the storagesystem according to further examples of the presently disclosed subjectmatter.

The volume-version data structure 200 also records data for snapshotoperations in which a source of the snapshot operations is a volume (aLU volume or a snapshot volume) from the volume-version data structure200. The data that is recorded for each one of the snapshot operationsincludes an indication of a connection between a respective targetvolume and a source volume, and a timestamp for the snapshot operation.As an example, and as is shown for illustration in FIG. 2, the timestampfor the snapshot operation can be a logical timestamp, however it shouldbe noted that any other appropriate timing marker can be used.

As mentioned above, according to examples of the presently disclosedsubject matter, the snapshot layer can hold a plurality ofvolume-version data structures for unrelated sets of volumes. Further byway of example, each volume-version data structure can include data inrespect of LU volume(s) and snapshot volumes which belong to therespective set of volumes and data in respect of snapshot operationswhich are associated the volumes in the set.

According to examples of the presently disclosed subject matter, a newvolume-version data structure can be created when a new LU volume iscreated (e.g., by a volume create command). In case a tree datastructure is used, as is the case in FIG. 2, the LU volume can be, forexample, located at the root of the volume-version tree. However, itshould be appreciated, that according to examples of the presentlydisclosed subject matter, in certain storage system implementations, aLU volume is not limited to appear only at the root of a volume-versiondata structure. For example, a certain LU volume can be restored at somepoint, from one of the snapshot LUs, and in such a case, the restorationoperation can be recorded in the volume-version data structure in amanner similar to the recordation of snapshot operation, and in respectof the restoration operation, a volume entry can be added to thevolume-version data structure, together with a timestamp for therestoration operation and an indication of a connection between therespective target volume (which is in this case the restored LU volume)and a source volume (in this case it is a snapshot volume). Further inresponse to a restore command, the volume entry for the volume which isbeing restored by the restore operation may be removed from thevolume-version data structure. It would be further appreciated that insome examples of the presently disclosed subject matter, in case of arestoration operation, the target volume does not receive a new volumeID, and rather the volume ID of the volume that is being restored isused for the target volume. Thus, in case the target volume of therestoration operation is a LU volume, the target volume would berecorded in the volume-version data structure with the volume ID of therestored LU volume. In a similar manner, in case the target volume ofthe restoration operation is a certain snapshot volume, the targetvolume would be recorded in the volume-version data structure with thevolume ID of the restored snapshot volume.

According to examples of the presently disclosed subject matter, when asnapshot volume is created (e.g., by a snapshot create command), data inrespect of the new snapshot volume is added to the volume-version datastructure. By way of example, the new snapshot volume can be a snapshotof the LU volume at the root of the volume-version data structure (theLU volume is the source of the snapshot operation) or a snapshot of asnapshot volume from the volume-version data structure, which in turncan be a snapshot of another snapshot volume, etc. When a snapshotvolume which is associated with the volume-version data structure iscreated, a new node or entry representing the new snapshot volume (thetarget snapshot volume) is added to the volume-version data structure.For example, a volume ID representing the new snapshot volume can beadded to the volume-version data structure. Additionally in response tothe creation of the new snapshot volume, a timestamp for the snapshotoperation and an indication of a connection between the target snapshotvolume and the source volume is added to the volume-version datastructure. For example, in response to a command to create a snapshotreferencing a certain source volume (the source volume can be a LUvolume or another snapshot volume) and a target volume, the followingset of operations can be implemented: I/Os to the source volume can besuspended; a timestamp or counter can be incremented; data structureswhich are related to the snapshot operation can be updated, such as therespective volume-version data structure and possibly also therespective chunk version data structure; and the lock on the sourcevolume can be released allowing the I/Os to the source volume to resume.

For example, in a tree format volume-version data structure such asshown in FIG. 2, when a snapshot command is received at the storagesystem, say a create snapshot command in which the source volume is LU1and the target volume is LU2, a new node is added to the appropriatevolume-version data structure. Thus for example, in FIG. 2, a node withthe volume ID LU2 is added to the volume-version data structure 200. Inthe example of FIG. 2, the node for LU2 is added to the tree formatvolume-version data structure 200 by adding an edge from the sourcevolume, which is LU1 in this case, to the new node LU2. This edgeindicates that connection between LU2 (the snapshot target volume) andLU1 (the snapshot source volume). Further in the example of FIG. 2, thetimestamp representing the logical time count that was obtained for thesnapshot operation, which is in this case T_(—)7, is assigned to theedge connecting LU2 to LU1.

According to examples of the presently disclosed subject matter, thesnapshot layer is also capable of removing a volume. According toexamples of the presently disclosed subject matter, removing a volumecan involve, inter-alia, removing a node from the volume-version datastructure, or marking the volume entry which is associated with thevolume that is to be removed with an indication that the correspondingvolume was removed.

According to examples of the presently disclosed subject matter, in casea certain volume is removed, the snapshot layer can be capable offreeing or updating any data or metadata that was kept in the storagesystem for the volume which was removed. This operation can beasynchronous with the removal of the volume from the volume-version datastructure and can be carried out using any suitable reinstatement methodknown in the art. According to examples of the presently disclosedsubject matter, various storage space redemption processes can beimplemented in the storage system to determine when a volume is nolonger in use (e.g., it has been deleted and there are no remainingreferences to the deleted volume), and the data in the volume versiondata structure in respect of the deleted volume can be deleted. By wayof example, according to one possible implementation, the usage state ofthe various volume entries in the volume version data structure can bemonitored and only when a certain volume is not referenced any more, thevolume version data structure can be updated and the volume entry whichis no longer in use can be deleted. Further by way of example, prior tothe volume version data structure update the volume version entry forthe deleted volume can exist (following the volume remove/deletecommand) but can only be referenced indirectly by reads from othersnapshots that are directed to the deleted volume.

According to examples of the presently disclosed subject matter, when arevert or a restore command is received at the storage system, a newnode can be added to the volume version data structure, and the new nodereceives the volume ID of a previously existing node in the volumeversion data structure. The new node can be connected to the node thatis associated with the source volume of the revert or restore operation.Additionally in response to the revert or restore operation, a timestampfor the revert or restore operation and an indication of a connectionbetween the target volume and the source volume is added to thevolume-version data structure. According to examples of the presentlydisclosed subject matter, further in response to the revert or restoreoperation, the previous version of the volume that is associated withthe volume ID which is the target of the revert or restore operation ismarked as being invalid or removed.

It would be further appreciated that examples of the presently disclosedsubject matter are not limited to the particular data-structure formatillustrated in FIG. 2, and rather any suitable data structure can beused for storing the volume-version data mentioned herein.

It will also be appreciated that the snapshot layer 20 of the storagesystem 100 can include a volume-version controller (not shown) which canbe associated with the volume-version data structure 122, and can beadapted to control various aspects of the operation thereof. For examplethe volume version controller can be configured to create a newvolume-version data structure 122, for example in response to a volumecreate command, and can be responsive to update an existingvolume-version data structure 122 when a create snapshot command isreceived. The volume-version controller can also be responsive to othercommands which involve affecting changes (including creation anddeletion) with respect to a volume-version data structure, including forexample, a volume restore command and a volume delete or a volume removecommand. In further examples of the presently disclosed subject matter,the functions of the volume-version controller can be carried out by theIO handler 126. The IO handler 126 can include or can be associated withone or more processors, and can include or be associated with one ormore memory units. According to examples of the presently disclosedsubject matter, the IO handler 126 can be configured to utilize theprocessing and memory resources to carry out the snapshot layerfunctions and operations described herein.

In addition the volume-version data structure 122, the snapshot layer 20holds chunk-version data structures. For the purpose of illustration asingle chunk-version data structure is shown in FIG. 1, and isreferenced by the numeral 124. Each chunk-version data structure 124 isuniquely associated with a specific snapshot-layer chunk. As mentionedabove, each snapshot-layer chunk can be associated with a unique volumeID and a logical chunk address (LCA) combination, and this combinationcan be used as the key for the respective chunk-version data structure.Thus, for a plurality of different logical chunk addresses of a givenvolume, a respective plurality of chunk-version data structures are keptin the storage system 100. Furthermore, for two LCAs in two differentvolumes (each of the different volumes has a different volume ID), twodifferent chunk-version data structures are kept, even if the LCAs ofthe two snapshot layer chunks are the same. This is because each chunkversion data structure is uniquely associated with a distinct snapshotlayer chunk, which is in turn associated with a unique volume ID and LCAcombination.

Each chunk-version data structure 124 holds one or more chunk-versiondata elements. Each chunk-version data element can be stored in any datacontainer in any suitable form or structure. Each chunk-version dataelement is associated with a specific chunk-version data structure 124.

Each chunk-version data element includes a timestamp at a correspondingwrite to the respective snapshot layer chunk and a pointer to storageresources in the storage layer 30 that are associated with thechunk-version data element. By way of example, a given chunk-versiondata structure 124 can have a plurality of chunk-version data elements,where each one of the plurality of chunk-version data elements is adistinct data container that, in addition to being associated with thesnapshot layer chunk with which the respective chunk-version datastructure 124 is associated, provides a timestamp and a reference or apointer to storage resources, where the timestamp indicates the time ofa corresponding write to the respective snapshot layer chunk, and thereference to the storage resources indicates the storage resource in thestorage layer 30 that were allocated and are used for storing therespective write data in the storage layer. Thus, according to examplesof the presently disclosed subject matter, given a certain volume ID andLCA combination and a certain timestamp at a write operation to thecorresponding snapshot layer chunk, data with regard to the storagelayer resources that were allocated and which are used for storing inthe storage layer the respective write data can be obtained. Moredetails about the chunk-version data structure 124 and about thechunk-version data elements are provided below.

According to examples of the presently disclosed subject matter, adedicated chunk-versioning controller (not shown) may be provided andcan be implemented as part of the snapshot layer 20 of the storagesystem 100 for controlling and/or carrying out the various functionsand/or operations involving the chunk-version data structures 124, or asis shown in FIG. 1 and according to further examples of the presentlydisclosed subject matter, the IO handler 126 can be configured tocontrol and/or carry out such functions and/or operations.

According to examples of the presently disclosed subject matter, thecentral controller 10 can be capable of controlling the operation of thestorage system 100 and its components. According to examples of thepresently disclosed subject matter, the central controller 10 caninclude a processor and computer memory (not shown) and can control theflow of data to and from each of the interface 40, the snapshot layer 20and the storage layer 30.

Reference is now made to FIG. 3, which is a flowchart illustration ofmethod of managing a storage system, according to examples of thepresently disclosed subject matter. It would be appreciated that themethod shown in FIG. 3 and described herein with reference thereto canbe, but is not necessarily, implemented by the storage system shown inFIG. 1.

According to examples of the presently disclosed subject matter, avolume-version data structure can be maintained in the snapshot layer ofthe storage system, including volume entries for an LU volume and forsnapshot volumes which are associated with the LU volume (block 305). Inassociation with each volume entry, a respective volume ID can bemaintained (block 310), and for each snapshot operation, a timestamp ofthe snapshot operation and an indication of a connection between atarget volume and a source volume of the snapshot operation can bestored in the volume-version data structure (block 315). As mentionedabove, in accordance with examples of the presently disclosed subjectmatter, within each volume-version data structure, each one of aplurality of volume IDs is unique, and in some implementations thevolume IDs can also be unique across the storage system.

According to examples of the presently disclosed subject matter, achunk-version data structure can also be maintained in the snapshotlayer of the storage system. As mentioned above, a plurality ofchunk-version data structures can be maintained, and each chunk-versiondata structure can be associated with a specific snapshot layer chunkwhich is referenced by a respective unique combination of a volume IDand a logical chunk address (block 320).

A chunk-version data structure can include one or more chunk-versiondata elements, each chunk-version data element including a timestamp ata corresponding write to the respective logical chunk address in therespective volume and a reference to storage layer resources which wereallocated for storing the respective write data in the storage layer(block 325).

More details with respect to the chunk-version data structure and thechunk-version data elements are now provided with reference to FIG. 4,which is a graphical illustration of a chunk-version data structure, andthe chunk-version data elements contained therein at two different timeinstants, according to examples of the presently disclosed subjectmatter. For the sake of illustration, the scenario depicted in FIG. 4corresponds with the volume-version data structure shown in FIG. 2. InFIG. 2 the volume which is associated with volume ID LU1 is created atT_(—)0. As mentioned above, in response to the volume create command,the snapshot layer can create a volume-version data structure and recordin the volume-version data structure the volume ID of the new volume(LU1). Since this is the first volume in the volume-version datastructure, LU1 is located at the root of the data structure (in case atree data structure is used). This operation can be managed andcontrolled by an IO handler (referenced 126 in FIG. 1), which isimplemented in the snapshot layer 20 or, in a further example, by adedicated volume-version controller.

According to the example scenario depicted in FIG. 4, when the counteris at T_(—)2, a write command is received in the storage system. Thewrite command references LU1 and LBAs 520-522. According to examples ofthe presently disclosed subject matter, each LBA is 8 bytes in size, andeach chunk is 128 kb in size. Thus, the snapshot layer determines thatthe write command is associated with LCA2 in volume LU1. According toexamples of the presently disclosed subject matter, the snapshot layer20 (e.g., using the IO handler) can determine whether there is anexisting chunk-version data structure for <LU1, LCA2>. If necessary, adata structure for LCA2 in volume LU1 can be created.

At instant 410, the chunk-version data structure for LCA2 in volume LU1includes a chunk-version data element with the timestamp T_(—)2 and areference to SRAs 512-767. According to examples of the presentlydisclosed subject matter, the chunk-version data structure for LCA2 involume LU1 can be created when the first write command referencing LCA2in volume LU1 (or associated therewith) is received at the storagesystem, however, in other examples the chunk-version data structure canbe created in advance, for example, when the volume (in this case LU1)is created.

The handling of write commands in the storage system is discussedfurther below, however, for the sake of illustration, it is to be notedthat at some point after the creation of LU1, a write command which isaddressed to LU1 may be received at the storage system, and in this casethe write command references LBAs 520-522 in LU1. Further by way ofexample, the snapshot layer 20 (e.g., the IO handler) can receive thewrite command and determine that it is associated with LCA2 in volumeLU1. Some details in respect to the handling of writes in the snapshotlayer are discussed below, however, as can be seen in FIG. 4, when it isdetermined that the write command is associated with the snapshot layerchunk that is referenced by the key <LU1, LCA2>, a chunk-version dataelement which is associated with the write command is included in therespective volume-version data structure (the one that is associatedwith the key <LU1, LCA2>), or in some cases an existing chunk versiondata element is overwritten with data related to the current writecommand, as will be further described below.

The chunk-version data element includes the timestamp T_(—)2, which isthe timestamp that was read from the counter when the write command wasreceived at the snapshot layer. According to examples of the presentlydisclosed subject matter, the counter is a logical counter which isincremented by operations concerning volumes, for example, by operationswhich affect or modify the data in the volume-version data structure.The chunk-version data element also includes a reference or a pointer tostorage layer blocks 720-728 which were allocated for storing therespective write data. The allocation of storage layer blocks for agiven write command and the mapping between logical blocks, snapshotlayer chunks and storage layer blocks is further discussed below.

Also shown in FIG. 4 by way of illustration, is the same volume-versiondata structure at a different time (instant 420). At time instant 420,the volume-version data structure associated with LCA2 in LU1 includestwo volume-version data elements. The first data element has thetimestamp T_(—)3 and references (or points to) SRAs 640-648. The seconddata element has the timestamp T_(—)8 and references SRAs 512-520.

The first chunk-version data element at time instant 420 (the one whichhas the timestamp T_(—)3), overwrote the chunk-version data elementwhich previously existed in the chunk-version data structure (which hadthe timestamp T_(—)2). The second chunk-version data element at timeinstant 420 (the one which has the timestamp T_(—)8) exists in thechunk-version simultaneously with the first chunk-version data element,and among these two chunk-version data elements an overwrite operationwas not appropriate. When the data in a previous (according to itstimestamp) chunk-version data element is needed by a snapshot volume (orvolumes) which is a direct or an indirect snapshot of the volume withwhich the chunk-version data structure (or the chunk-version dataelements) are associated, the previous chunk-version data element can bemaintained, and will not be overwritten by the chunk-version dataelement. The management of the chunk-version data structure, includingexamples of management logic that can be used to determine when tooverwrite existing chunk-version data elements, and when to retainmultiple chunk-version data elements in a given chunk-version datastructure, shall be further described below.

The chunk-version data structure at instant 420 serves as an example ofa chunk-version data structure which has a plurality of chunk-versiondata elements. It should be appreciated that a chunk-version datastructure can include any number of chunk-version data elements from 1to n, at any given instant. In some examples of the presently disclosedsubject matter, a chunk-version data structure which has no (0)chunk-version data elements can exist, for example when the operationwhich removes empty chunk version data structures (e.g., could bereferred to as a “chunk free” operation) is a lazy background operation.

According to examples of the presently disclosed subject matter, thechunk-version data structure, together with the volume-version datastructure, can be used to enable or support IO operations (or some IOoperations) in a storage system which implements snapshot volumes, aswill be further described below.

Reference is now made to FIG. 5, which is a block diagram illustrationof a storage system which includes a snapshot layer for supporting andmanaging snapshot volumes and data versioning. According to examples ofthe presently disclosed subject matter, the snapshot layer 520 caninclude, in addition to the 10 handler 526, the volume-version datastructure 122 and chunk-version data structure 124, a chunk-provisioningdata structure 522 and a free storage layer resources list 524.

According to examples of the presently disclosed subject matter, thefree storage layer resources list 524 can hold a list of the currentlyavailable storage layer addresses (SRAs), e.g., storage layer blocks,which can be allocated to snapshot layer chunks versions. The freestorage layer resources list 524 can be a dynamic list, and can providean up to date image of the availability of storage layer resources whichcan be allocated to snapshot layer chunks versions. Still further by wayof example, each entry in the free storage layer resources list 524 canrelate to a certain set of SRAs which corresponds in size to a snapshotlayer chunk. Thus, when a new version of a snapshot layer chunk iscreated, data in respect of a free chunk of storage layer resourceswhich corresponds in size to the snapshot layer chunk can be selectedfrom the free storage layer resources list 524, and can be allocated tothe newly created snapshot layer chunk version. As was explained above,according to some examples of the presently disclosed subject matter,and in some circumstances, a given snapshot layer chunk can have morethan one version (e.g., two, three, . . . , n versions), and differentstorage layer resources can be allocated to each different version ofthe same snapshot layer chunk.

For example, the storage layer chunks in the free storage layerresources list 524 can be enumerated (e.g., from 1 to n), with eachstorage layer chunk being associated with a corresponding set of storagelayer blocks of a certain size. Still further by way of example, theassociation between storage layer chunks and SRAs can be such that thefirst storage layer chunk is associated with the first set ofconsecutive SRAs of the predetermined size in the storage space, thesecond to the storage layer chunk is associated with the second set ofSRAs in the storage space, etc. In this configuration a mapping table ora mapping function are not needed to determine the relation betweenstorage layer chunks in the free storage layer resources list 524 andthe SRAs in the storage layer. However, in further examples, a mappingtable or function may be required, or alternatively the free storagelayer resources list 524 can use SRA ranges instead of storage layerchunks, thus directly linking snapshot layer chunks with correspondingSRAs.

It should be noted, that according to examples of the presentlydisclosed subject matter, that the storage layer blocks can beassociated with validity status indications, e.g., validity status bits,and for the storage layer blocks which are associated entries in thefree storage layer resources list 524 (e.g., with a storage layer chunkin the free storage layer resources list 524), the validity status bitshave an ‘invalid’ value, assuming that the free storage layer resourceslist 524 maintains data only for ‘free’ storage layer resources.

According to examples of the presently disclosed subject matter, whenstorage layer resources are allocated to a certain snapshot layer chunkversion, the free resource list 524 can be updated to reflect theallocation. Thus for example, in case storage layer chunks are used, theserial number which represents a free storage layer chunk can be removedfrom the free storage layer resources list 524 when that storage layerchunk is allocated to some snapshot layer chunk version (and thus ceasesto be free). In a further example, the record for the now allocatedstorage layer chunk can be kept in the free storage layer resources list524 after it is allocated, and an indication can be added in associationwith the allocated storage layer chunk record which indicates that thischunk is not free (or is allocated). Further by way of example, theindication can include a reference to the snapshot layer chunk versionto which the storage layer chunk was allocated. For example, thereference to the snapshot layer chunk version to which the storage layerchunk was allocated can include the respective volume ID and LCAcombination, and the timestamp that is associated with the respectivesnapshot layer chunk version.

It would be appreciated that the term list as used herein as part of theexpression free storage layer resources list is not intended to limit orbound the form or structure of the data container, and that any suitabledata container can be used to record, track and manage the data inrespect of the free storage layer resources.

According to examples of the presently disclosed subject matter, thechunk provisioning data structure 522 is capable of maintaining for eachallocated snapshot layer chunk a pointer or pointers to the storagelayer resources that were allocated to that snapshot layer chunk. Asmentioned above, according to examples of the presently disclosedsubject matter, the free storage layer resources list 524 can beimplemented as a list of storage layer resources which are available forallocation to snapshot layer chunks versions, and whenever storage layerresources are allocated, the corresponding data is removed from the freestorage layer resources list 524. The chunk provisioning data structure522 can be used to determine which storage layer resources wereallocated to which snapshot layer chunks versions.

As was also mentioned above, several different versions (e.g., two,three, . . . , n versions) of a given snapshot layer chunk can existsimultaneously, and the chunk provisioning data structure 522 can becapable of maintaining a pointer or pointers to the storage layerresources that were allocated to each version of a given snapshot layerchunk, in case there are multiple versions of the snapshot layer chunk.

Still further by way of example, the chunk provisioning data structure522 can be organized in the form of a hash table. The hash table key caninclude a combination of an LCA and a volume ID. Thus, according to themapping scheme described above, given a volume ID and LBA address, thehash table key that is used by the chunk provisioning data structure 522can be determined. According to examples of the presently disclosedsubject matter, each hash table entry that is associated with a givenkey can point to the chunk-version data structure that is associatedwith the respective <volume ID, LCA> combination.

Further by way of example, for each snapshot layer chunk version, thepointer or pointers to the storage layer resources which were allocatedto this snapshot layer chunk version are maintained in the correspondingchunk-version data element. Thus, when multiple versions (two, three, .. . , n versions) of the same snapshot layer chunk exist, the chunkprovisioning data structure 522 together with the chunk-version datastructures and the chunk-version data elements can be used to determinewhich storage layer resources are associated with each version of agiven snapshot layer chunk.

Further by way of example, when a write command is received at thestorage system, the IO handler (or any other control module) in thesnapshot layer can search the chunk-version data structure (e.g.,through the chunk provisioning data structure) as part of the handlingof the write command in the storage system. The search can return a nullresult, for example, when there is no current version of the snapshotlayer chunk for which storage layer resources were allocated. In such acase, according to examples of the presently disclosed subject matter,the IO handler can be configured to request an allocation of storageresources from the free storage layer resources list 524 for therespective snapshot layer chunk. It should be appreciated, thataccording to some examples of the presently disclosed subject matter,allocation of storage resources from the free storage layer resourceslist 524 can also be required when there are already one or moreversions of the relevant snapshot layer chunk, but an additional versionof the snapshot layer chunk is to be created.

It would be appreciated that searching the chunk provisioning datastructure 522 hash table for a chunk-version data structure that isassociated with a given <volume ID, LCA> key can yield relatively goodaccess times. Adding an additional key for a newly allocatedchunk-version data structure can also be relatively straightforward andefficient. In this regard, using a hash table for this applicationfavors performance over storage space. However, it should also beappreciated that examples of the presently disclosed subject matter arenot limited to this particular form of data structure, and that thechunk provisioning data structure 522 and the manner by which it isorganized and used with the chunk-version data structures can beotherwise implemented.

There is now provided a description of the use of the chunk provisioningdata structure 522 and the free storage layer resources list 524,according to one example of the presently disclosed subject matter. Inthis example, it assumed that the storage layer implements a 512 bytesaccessible sectors size, and 4 KB management segments (e.g., this is thegranularity of the validity data), and the snapshot layer implements 128KB snapshot layer chunks. The chunk provisioning data structure 522 thusholds a list of free 128 KB storage layer chunks, and the IO handler 526can obtain such 128 KB storage layer chunks for newly allocated snapshotlayer chunks of for new versions of an existing snapshot layer chunks.The actual writing of the data in the storage layer is carried out insegments.

In one example, assume a write command is received at the storage systemreferencing blocks 520-535 in LU1 (SCSI address). The IO handlerconverts the SCSI address by rounding down the LBA to LCA (logical chunkaddress). Assuming that storage resources were not previously allocatedto this snapshot layer chunk, or that the allocated resources should notbe overwritten by the current write, the chunk provisioning datastructure 522 may return a null. In response to the null from the chunkprovisioning data structure, the IO handler 526 may request a chunkallocation from the free storage layer resources list 524. By way ofexample, the free storage layer resources list 524 may return areference to a free storage layer chunk. The free storage layer chunkcan be simply a number—in case the IO handler is preconfigured todetermine a set of storage segments from a chunk number. Anotheralternative is that the free storage layer resource list returns thesegments addresses or the storage layer block addresses which areassociated with the selected storage layer chunk. Any other scheme forpointing towards the storage layer resources can be used. Still furtherby way of example, the IO handler 526 may send an insert command to thechunk provisioning data structure 522. For example a command similar tothe following can be used: insert key=“<LU1, LCA2>, value=#3”,instructing the chunk provisioning data structure 522 to add to theappropriate chunk-version data structure 124 a chunk-version dataelement which references the storage layer chunk referenced “3”. Furtherby way of example, the handler 526 can now write the data to theallocated storage layer chunk. The 10 handler 526 may compute thesegments in the storage layer which correspond to the storage layerchunk that was allocated for the write command. For example, the handler526 can be adapted to determine the relative location of the segmentswhich are associated with the logical block addresses referenced in thewrite command within the chunk (snapshot layer chunk or storage layerchunk). For example, assuming 128 kb chunks, 4 kb segments, and 512bytes LBAs, for blocks 520-535, when the allocated chunk is #3 (thethird chunk counted from the start of block 1), segments (1,2) in chunk#3 should be used, which translates to segments 97 and 98 in the storagelayer 30. Upon receiving the write command, the storage layer 30 willwrite the data to the specified segments and will mark the relevantsegments as valid (each segment write and validity status update arecarried out as a single atomic operation). Once the write is completedsuccessfully, the storage layer 30 sends an ACK to IO handler 526 in thesnapshot layer 520, and in response the IO handler 526 sends an ACK tothe initiator of the write command.

Further by way of illustration, another example of the use the chunkprovisioning data structure 522 and the free storage layer resourceslist 524 is now provided with reference to scenario illustrated in FIG.4. In response to the write command at instant 410 (associated withtimestamp T_(—)2), the IO handler converts the SCSI address by roundingdown the LBA to LCA (logical chunk address), and it is thus determinedthat the write command is associated with LCA2 in LU1. In this example,it is assumed that the chunk provisioning data structure returned a nullfor this write command indicating that storage resources were notpreviously allocated to this snapshot layer chunk, and in response, thefree storage layer resources list allocated free storage layer blocks.In FIG. 4, the storage layer resources are allocated as a set ofconsecutive storage layer blocks which correspond in size to asnapshot-layer chunk. Further by way of example, for the chunk-versiondata element shown at instant 410 (associated with timestamp T_(—)2),the free storage layer resources list allocated blocks 512-767 (256blocks, each 512 bytes and size, for a 128 KB chunk). The IO handlerupdated the chunk provisioning data structure, and at instant 410, thechunk-version data structure includes a data element that includes atimestamp that indicates the state of the counter at the time when therespective write command was received (in this case T_(—)2) and areference to the storage layer resources which are associated with thischunk-version data element (blocks 512-767).

Further with reference to the example in FIG. 4, when the counter isshowing T_(—)3 another write which is also associated with <LU1, LCA2>was received and processed by the snapshot layer. In the case of thiswrite command, no further allocation of resources from the free storagelayer resources list was necessary, because it was determined that thechunk-version data element which is associated with the write attimestamp T_(—)3 should update and/or overwrite the chunk-version dataelement that is associated with a write at timestamp T_(—)2, and thestorage layer resources that were allocated to the chunk-version dataelement that was associated with timestamp T_(—)2 should be referencedby the chunk-version data element which is associated with timestampT_(—)3. By way of example, the write data of the write command receivedwhen the counter was at T_(—)3 can be used to update and/or overwritethe data in the storage layer resource that were allocated to theoverwritten chunk-version data element. Further by way of example, ifthe write data in the write command that is associated with T_(—)3overlaps with some (or all) of the write data that was previously storedin the storage resources that are allocated to the chunk-version dataelement, the new write data will overwrite the existing data. Anynon-overlapping data is used to update the allocated storage resources(there is no old data there to overwrite).

Still further with reference to the example in FIG. 4, another writewhich is associated with <LU1, LCA2> was received and processed by thestorage system. This write command was received when the counterregistered T_(—)8, and so it received the timestamp T_(—)8. In the caseof this write command, the IO handler determined that an overwrite ofthe existing chunk-version data element (the one associated with T_(—)3)is not appropriate, and a new chunk-version data element should becreated. The IO handler thus requests an allocation of storage layerresources from the free storage layer resources list. By way of example,for the chunk-version data element that is associated with timestampT_(—)8, the free storage layer resources list allocated blocks 768-1023.The IO handler updates the chunk provisioning data structure, and atinstant 420, the chunk-version data structure includes in addition tothe chunk-version data element associated with the timestamp T_(—)3, achunk-version data element that is associated with timestamp T_(—)8, andeach of the two chunk-version data elements is associated with adifferent set of storage layer blocks in a size which corresponds to asnapshot layer chunk. It would be appreciated that the size of thedifferent units, the particular timestamp values, the particularaddresses and the number of chunk-version data elements in thechunk-version data structure are all provided by way of non-limitingexample, and can be substituted with any appropriate units, values,addresses or numbers.

According to a further aspect of the presently disclosed subject matter,there is provided a method of handling writes in a storage system.According to examples of the presently disclosed subject matter, themethod of handling writes in a storage system can include: maintaining avolume-version data structure including volume entries for a LU volumeand for snapshot volumes associated with the LU volume, wherein for eachvolume a respective volume ID is kept, and for each snapshot a timestampfor the snapshot operation and an indication of a connection between arespective target volume and a source volume are kept; maintainingchunk-version data structures, each of which is uniquely associated witha respective snapshot-layer chunk, and each of which includes one ormore chunk-version data elements, wherein each chunk-version dataelement includes a timestamp at a corresponding write to the respectivesnapshot-layer chunk and a pointer to storage resources associated withthe chunk-version data element; when a write command that is associatedwith an existing chunk-version data structure is received, reading atimestamp of a last chunk-version data element in the existingchunk-version data structure, and if there is a snapshot operation whosesource is the volume referenced in the write command and this snapshotoperation is associated with a timestamp that is later than thetimestamp of the last chunk-version data element, maintaining the lastchunk-version data element and adding to the existing chunk-version datastructure a new chunk-version data element for the write command.According to further examples of the presently disclosed subject matter,if there is no snapshot operation whose source is the volume referencedin the write command, or if all snapshot operations (e.g., one or two orN snapshot operations) whose source is the volume referenced in thewrite command have a timestamp that is earlier than or equal to thetimestamp in the last chunk-version data element, the IO handler isconfigured to determine that the last chunk-version data element in theexisting chunk-version data structure can be overwritten.

For sake of clarity, it should be noted that FIGS. 1-5 and thedescription thereof which was provided above, should be regarded,according to examples of the presently disclosed subject matter, as partof the description of the method of handling writes in a storage system,which is now described. It should be further noted that the method ofhandling writes in a storage system can be implemented in a storagesystem that is designed and managed according to the disclosure providedhereinabove, including with reference to FIGS. 1-5. In further examplesof the presently disclosed subject matter, the system for handlingwrites in a storage system is not limited by some or all of theteachings provided above, in particular with reference to FIGS. 1-5.

It should also be noted that in some examples of the presently disclosedsubject matter, the method of handling writes in a storage system can beimplemented in conjunctions with the teachings provided above, inparticular with reference to FIGS. 1-5. In further examples of thepresently disclosed subject matter, the method of handling writes in astorage system is not limited by some or all of the teachings providedabove, in particular with reference to FIGS. 1-5.

Reference is now made to FIG. 6, which is a flowchart illustration of amethod of handling writes in a storage system, according to examples ofthe presently disclosed subject matter. Further by way of example, themethod laid out in FIG. 6 can be implemented by the storage system shownin FIG. 1 and described above. Still further by way of example, themethod laid out in FIG. 6 can be implemented by the storage system shownin FIG. 5 and described above. Accordingly, by way of non-limitingexample, the description of FIG. 6 is made with additional reference tothe storage system shown in FIG. 1 or with reference to the storagesystem shown in FIG. 5.

According to examples of the presently disclosed subject matter, asnapshot layer 20 of the storage system 100 can maintain a volumeversion data structure 122, that includes volume entries for an LUvolume and for snapshot volumes associated with the LU volume, whereinfor each volume a respective volume ID is kept, and for each snapshot atimestamp for the snapshot operation and an indication of a connectionbetween a respective target volume and a source volume are kept (block605). By way of example, a tree data structure can be used. Further byway of non-limiting example, the volume version data structure 122 canbe implemented in a manner that is similar to the one illustrated inFIG. 2 and described above with reference to FIG. 2, however any othersuitable data structure can be used.

Resuming the description of FIG. 6, according to examples of thepresently disclosed subject matter, the snapshot layer 20 of the storagesystem 100 can further maintain chunk-version data structures 124, eachof which is associated with a particular snapshot-layer chunk, and eachof which includes one or more chunk-version data elements.

According to examples of the presently disclosed subject matter, eachchunk-version data structure can be associated with and referenced by aunique volume ID and logical chunk address combination, and eachchunk-version data element includes a timestamp at a corresponding writeto the respective logical chunk address and a pointer to storageresources associated with the chunk-version data element (block 610).Examples of chunk-version data structures, and chunk-version dataelements and the use thereof in managing of snapshot volumes and dataversioning were discussed above with reference to FIGS. 1-5 and shouldbe regarded, according to examples of the presently disclosed subjectmatter, as part of the description of the method of handling writes in astorage system.

At some point, a write command can be received by the storage system100. As mentioned above, the write command can be received at thestorage system 100 (block 615) through an interface 40, and is passed(possibly after some processing at the interface 40) to the snapshotlayer 20.

According to examples of the presently disclosed subject matter, uponreceiving the write command at the snapshot layer 20, the IO handler126, which is configured to control the processing of the write commandin the snapshot layer 20, can be configured to determine whether thewrite command is associated with an existing chunk-version datastructure (block 620).

According to examples of the presently disclosed subject matter, eachwrite command references a certain logical address. Further by way ofexample, each write command references a certain logical unit (orvolume) and a logical block or blocks to which it relates. For example,assuming the storage system supports the SCSI storage protocol, a writecommand would include a reference to a logical unit (LU) and a LBA or arange of LBAs where the data in the write command should be saved. Sincethe address provided to the storage system is a logical address, theactual storage location of the data can have a different address, butthe storage system maintains a mapping of logical storage addresses(e.g. snapshot-layer chunks) and the corresponding actual storagelocations where the data is stored in the storage system. As mentionedabove, more than one virtualization level can be implemented, in whichcase additional layer of logical addresses are implemented and furthermapping functions, table or other means are used.

As mentioned above, the snapshot layer 20 can implement snapshot layerchunks. Each snapshot layer chunk can be referenced by a uniquecombination of a LCA address (e.g., a number) and volume ID (forexample, the logical unit number referenced in the write command). Thesnapshot layer 20 is configured to translate LBAs in a given volume to arespective LCA (or LCAs) in the volume. As mentioned above, for example,the LBAs can be numerical and range from 1-n, and the corresponding LCAcan be derived by rounding down of the LBA address to the nearest LCA.

It would be appreciated that a given write command can reference alogical address range which is associated with more than one snapshotlayer chunk (e.g. two, three, . . . n snapshot layer chunks). It shouldbe noted that the process described herein can be implemented withrespect to the entire logical address range, by repeating the processfor each one of the snapshot layer chunks with which the write commandis associated, or carrying out multiple instances of the process inparallel. For simplicity, and by way of non-limiting example, the belowdescription assumes that the write command is associated with onesnapshot layer chunk.

According to examples of the presently disclosed subject matter, as partof determining whether the write command is associated with an existingchunk-version data structure, the IO handler 126 can search for achunk-version data structure 124 that is associated with the volume IDand with the logical chunk address with which the write command isassociated. For example, in case each chunk-version data structure 124in the snapshot layer 20 is associated with a <volume ID, LCA>combination, the IO handler 126 can search for a chunk-version datastructure 124 that is associated with the <volume ID, LCA> with whichthe write command is associated.

Still further by way of example, the storage system implementing themethod of handling writes can have a structure that is generally similarto the storage system 500 shown in FIG. 5 and described above withreference thereto. The snapshot layer 20 of the storage system 500 canfurther include a chunk provisioning data structure 522, in which case,in response to receiving a write command, the IO handler 126 can beconfigured to search the chunk provisioning data structure 522 todetermine whether the write command is associated with an existingchunk-version data structure. As described above, the chunk provisioningdata structure 522 can, for example, be implemented as a hash table inwhich the keys are unique <volume ID, LCA> combinations which areassociated with snapshot layer chunks for which storage resources in thestorage layer 30 were allocated. By way of example, at least in theimplementation of the storage system 500 shown in FIG. 5, an existingchunk-version data structure is one for which storage resources in thestorage layer 30 were allocated. Thus, according to examples of thepresently disclosed subject matter, to determine whether a given writecommand is associated with an existing chunk-version data structure, theIO handler 526 can search the hash table 522 using the volume ID and thelogical chunk address with which the write command is associated. Yetfurther by way of example, if the search returns ‘null’ or acorresponding result, the IO handler 526 can conclude that the writecommand is not associated with an existing snapshot-layer chunk.

According to examples of the presently disclosed subject matter, if theIO handler 526 concludes that the write command is not associated withan existing snapshot layer chunk, the IO handler 526 can be configuredto create a new chunk-version data structure 124 and a chunk-versiondata element corresponding to the write command (block 625). Thechunk-version data structure 124 will be associated with the volume IDand LCA with which the write command is associated and the volumeversion data elements shall include a timestamp which corresponds to thestate of the counter at the time when the write command was received andthe storage-layer resources that were allocated for this version of thesnapshot-layer chunk. The IO handler 526 can then proceed to instructthe storage layer 30 to write the write data to the storage-layerresources that were allocated for the write command (block 695).

The creation of a new chunk-version data structure 124 and a chunkversion data element were discussed above. In brief, and by way ofexample, the IO handler 526 can be configured to determine with whichsnapshot layer chunk the write command is associated, and can add achunk-version data structure 124 whose key is associated with thesnapshot layer chunk with which the write command is associated. Forexample, the chunk-version data structure can have a key that iscomprised of a combination of the volume ID and the logical chunkaddress of the snapshot-layer chunk.

The IO handler 526 can then add in connection with the write command anew chunk-version data element to the new chunk-version data structure124. The chunk-version data element includes a timestamp whichcorresponds to the state of the counter at the time when the writecommand was received and a pointer to storage layer resources that wereallocated for storing the write data in the storage layer 30.

According to examples of the presently disclosed subject matter, as wasmentioned above, the storage system 500 can include a chunk-provisioningdata structure 522 and a free storage layer resources list 524, and whena write command is received at the storage system 500, the IO handler526 can be configured to search for an existing chunk-version datastructure 124 that is associated with the write command using thechunk-provisioning data structure 522. If storage layer resources werepreviously allocated to the snapshot layer chunk with which the writecommand is associated, the chunk-provisioning data structure 522 willinclude an entry for that snapshot layer chunk, otherwise, the searchwill return a null result, indicating that the respective snapshot layerchunk does not exist, or is not initiated or allocated yet.

According to examples of the presently disclosed subject matter, in casethe search returned a null, the IO handler 526 can be configured torequest the free storage layer resources list 524 to allocate resourcesto the snapshot layer chunk with which the write command is associated.The IO handler 526 can be configured to add to the chunk provisioningdata structure 522 an entry for the snapshot layer chunk with which thewrite command is associated, and create a new chunk-version datastructure 124 which is associated with the new key in the chunkprovisioning data structure 522. The IO handler 126 can be configured toadd to the chunk-version data structure 124 a chunk-version data elementthat includes a timestamp that indicates the state of the counter at thetime when the write command was received and a pointer to the storageresources that were allocated for storing the write data in the storagelayer 30.

Returning now to the description of FIG. 6, in case at block 620 it wasdetermined that there is an existing chunk-version data structure 124for the snapshot layer chunk that is associated with the write command,the IO handler can be configured to read a timestamp in a lastchunk-version data element in the existing chunk-version data structure124 (block 630).

The IO handler 526 can then be configured to determine whether there wasa snapshot in which the volume referenced in the write command is asource volume and in which the timestamp associated with the snapshotoperation is later than the timestamp in the last chunk-version dataelement (block 635). According to examples of the presently disclosedsubject matter, in order to determine whether there was a snapshot inwhich the volume referenced in the write command is a source volume, andif such a snapshot exists, whether the timestamp associated with such atleast one such snapshot operation is later than the timestamp in thelast chunk-version data element, the IO handler 526 can search thevolume-version data structure 122 for volume entries which are connectedto the volume referenced in the write command, where the volumereferenced in the write command is a source of a snapshot operation, andwhere the timestamp for the snapshot operation is later than thetimestamp in the last chunk-version data element. For example, assumingvolume-version data structure 122 has a tree format, the search can becarried out with respect to child nodes of the node that is associatedwith the volume referenced in the write command (which can be regardedas the parent node), and the timestamp on each connector, which connectsthe parent node and its children node, is read to determine if there isa snapshot operation where the timestamp for the snapshot operation islater than the timestamp in the last chunk-version data element.

According to examples of the presently disclosed subject matter, in caseit is determined that there is no record (in the volume-version datastructure) of a snapshot operation where the source volume is the volumereferenced in the write command, and where the timestamp for thesnapshot operation is later than the timestamp in the last chunk-versiondata element in the chunk-version data structure 124 that is associatedwith the snapshot layer chunk with which the write command isassociated, then the IO handler 526 can be configured to overwrite thelast chunk-version data element in the chunk-version data structure 124that is associated with the snapshot layer chunk with which the writecommand is associated (block 640). It would be appreciated that in casethere is not any snapshot (at all) in which the volume referenced in thewrite command is a source volume, the process also proceeds from block635 to block 640.

In further examples of the presently disclosed subject matter, inconnection with determining whether a chunk-version data element shouldoverwrite an existing volume-version data structure or not, the IOhandler 526 can be configured to search for a snapshot operation whosesource is the volume referenced in the write command, and if there is nosuch snapshot (a snapshot whose source is the volume referenced in thewrite command), or if all snapshot operations whose source is the volumereference in the write command have a timestamp that is earlier than orequal to the timestamp in the last chunk-version data element in therespective chunk-version data structure, the IO handler 526 can beconfigured to determine that the last chunk-version data element can beoverwritten.

According to examples of the presently disclosed subject matter, theoverwrite operation can involve updating the storage resources that wereallocated to the last chunk-version data element with the write data inthe current write command, and optionally updating the timestamp in thelast chunk-version data element with the timestamp that indicates thestate of the counter at the time when the current write command wasreceived. As mentioned, in case the last chunk-version data element isoverwritten, there is no need to allocate further storage layerresources, and the storage layer resources that were previouslyallocated to the last chunk-version data element can be used for storingthe new write data. By way of example, the write data can be used toupdate and/or overwrite the data in the storage layer resources whichwere allocated to the overwritten chunk-version data element.

According to examples of the presently disclosed subject matter, ifhowever at block 635 it was determined that there was a snapshot inwhich the volume referenced in the write command is a source volume andin which the timestamp associated with the snapshot operation is laterthan the timestamp of the last chunk-version data element, the IOhandler 126 can be configured to maintain the last chunk-version dataelement in the chunk-version data structure that is associated with thewrite command, and the IO handler 126 can be configured to add to thechunk-version data structure a new chunk-version data element for thecurrent write command (block 645).

According to examples of the presently disclosed subject matter, in caseblock 645 is implemented, the IO handler can be configured to requestallocation of storage layer resources, for example, through the freestorage layer resources list 524. The allocated storage layer resourcesare then removed from the free storage layer resources list. Asmentioned above, the allocated storage layer resource can be a list or arange of storage layer block, a segment or segments of storage layerblock or a storage layer chunk. In case storage layer blocks or astorage layer segment or segments are allocated, the allocation cancorrespond to the extent of write data in the write command (the storagespace can be less than the full size of snapshot layer chunk), or theallocation of the storage layer block or segments can be in an extentwhich corresponds to the full size of a snapshot layer chunk.

The IO handler 126 can be configured to add a new chunk-version dataelement to the chunk-version data structure with which the write commandis associated. The new chunk-version data element includes a referenceto the storage layer resources that were allocated to the new version ofthe snapshot layer chunk, and the timestamp which represents the stateof the counter at the time when the write command was receive, e.g., acurrent timestamp. It would be appreciated that the new chunk-versiondata element becomes the last chunk-version data element in therespective chunk-version data structure.

Following the addition of the new chunk-version data element to thechunk-version data structure with which the write command is associated,the IO handler 126 can proceed to instruct the storage layer 30 to writethe write data in the storage resources that were allocated for thewrite command (block 695).

It would be appreciated, that the new chunk-version data element for thecurrent write command will become the last chunk-version data element inthe chunk-version data structure with which the write command isassociated.

There is now provided a description of the handling by a storage systemin accordance with examples of the presently disclosed subject matter ofa series of write commands. The following description can be used as anexample of an implementation of the method of handling writes in astorage system, according to examples of the presently disclosed subjectmatter, which was described above including with reference to FIG. 6.

Reference is now made to FIG. 7, which is a graphical illustration of achunk-version data structure, and the chunk-version data elementscontained therein, at different time instants, according to examples ofthe presently disclosed subject matter. For the sake of illustration,the scenario depicted in FIG. 7 corresponds with the volume-version datastructure shown in FIG. 2. In addition, the chunk-version data structureillustrated in FIG. 7 can be implemented by a storage system such as theone illustrated in FIG. 1 and described above. The chunk-version datastructure illustrated in FIG. 7 can also be implemented by a storagesystem such as the one illustrated in FIG. 5 and described above. Itwould be appreciated that the chunk-version data structure illustratedin FIG. 7 can be implemented by other suitably structured and configuredstorage systems and is not necessarily limited to the specific storagesystem examples used herein.

In FIG. 2 the volume which is associated with volume ID LU1 is createdat T_(—)0. As mentioned above, in response to the volume create command,the snapshot layer 20 can create a volume-version data structure andrecord in the volume-version data structure the volume ID of the newvolume (LU1). Since this is the first volume in the volume-version datastructure, LU1 is located at the root of the data structure (in case atree data structure is used). This operation can be managed andcontrolled by an IO handler (referenced 126 in FIG. 1), which isimplemented in the snapshot layer 20 or, in a further example, by adedicated volume-version controller.

According to the example scenario depicted in FIG. 7, at time instant702 a write command is received and processed in the storage system. Asmentioned above, the storage system can implement a clock, a timer, or acounter which can be used to provide a timestamp for volume operationsor for any other events of interest in the storage system. By way ofexample, at time instant 702, when the write command that is received,the counter state is T_(—)2 and so the write command is associated withthe logical timestamp T_(—)2. Further by way of example, the writecommand at instant 702 references LU1 and LBAs 520-522. According toexamples of the presently disclosed subject matter, each LBA is 8 bytesin size, and each chunk is 128 kb in size. Thus, the snapshot layerdetermines that the write command is associated with LCA2 in volume LU1.According to examples of the presently disclosed subject matter, thesnapshot layer 20 (e.g., using the IO handler 126) can determine whetherthere is an existing chunk-version data structure for <LU1, LCA2>. Ifnecessary, a data structure for LCA2 in volume LU1 can be created.

At instant 702, the chunk-version data structure for LCA2 in volume LU1includes a chunk-version data element with the timestamp T_(—)2 and areference to SRAs 512-767. According to examples of the presentlydisclosed subject matter, the chunk-version data structure for LCA2 involume LU1 can be created when the first write command referencing LCA2in volume LU1 (or associated therewith) is received at the storagesystem 100, however, in other examples the chunk-version data structurecan be created in advance, for example, when the volume (in this caseLU1) is created.

According to examples of the presently disclosed subject matter, thesnapshot layer 520 (e.g., using the IO handler 526) can receive thewrite command and determine, for example according to the mapping schemethat was described above, that the logical address referenced in thewrite command is associated with LCA2 in volume LU1. It would beappreciated that the aforementioned mapping scheme is merely an example,and that other mapping schemes can be devised an implemented in thestorage system according to examples of the presently disclosed subjectmatter.

As was described above, when an incoming write command is received atthe snapshot layer 520, the IO handler 526 (or any other snapshot layercontroller) can be configured to consult a chunk provisioning datastructure 522 to determine whether storage layer resources wereallocated to the snapshot layer chunk with which the write command isassociated. For the sake of illustration, it is to be noted that at somepoint after the creation of LU1, in this case at time instant 702, awrite command which is addressed to LU1 may be received at the storagesystem, and in this case the write command references LBAs 520-522 inLU1. Further by way of example, according to the mapping scheme that isimplemented by the IO handler 526, it is determined that the writecommand is associated with snapshot-layer chunk referenced by LCA2 inthe volume LU1.

The IO handler 526 (or any other snapshot layer controller) than usesthe key <LU1, LCA2> to search the chunk provisioning data structure 522to determine whether storage layer resources were allocated to thesnapshot layer chunk with which the write command is associated.According to the scenario of FIG. 7, the data in the chunk provisioningdata structure 522 indicates that the snapshot-layer chunk associatedwith the key <LU1, LCA2> has not been provisioned yet. Accordingly, thehandler 526 can request allocation of storage layer resources for thissnapshot layer chunk from the free storage layer resource list 524.Further by way of example, according to the scenario in FIG. 7, the SRAs512-767 are allocated for the snapshot layer chunk <LU1, LCA2>. Stillfurther by way of example, the chunk-provisioning data structure 522 isupdated to indicate that the snapshot layer chunk <LU1, LCA2> wasprovisioned, and a chunk-version data structure 122 is created for thesnapshot layer chunk <LU1, LCA2>. Further by of example the entry in thechunk-provisioning data structure 522 which is associated with <LU1,LCA2> includes a pointer to the chunk-version data structure 122 that isassociated with the snapshot layer chunk with which the <LU1, LCA2> isassociated.

According to examples of the presently disclosed subject matter, furtherin response to the allocation of the SRAs to the snapshot layer chunk, achunk-version data element 124 is added to the chunk-version datastructure 122. The timestamp T_(—)2 that was read from the counter whenthe respective write command was received (or at any other predefinedtime when the write command is processed) is included in thechunk-version data element, as well as the storage layer resources thatwere allocated for the current version of the snapshot layer chunk,which are in this case SRAs 512-767.

The IO handler 126 can then instruct the storage layer to write thewrite data in the SRAs that were allocated for the current version ofthe snapshot layer chunk.

Next, at instant 704, the state of the chunk-version data structure thatis associated with the snapshot layer chunk <LU1, LCA2> is shown, aftera further write command that is associated with this chunk is receivedand processed. The second write command to the snapshot layer chunk<LU1, LCA2> is received when the logical counter is at T_(—)3. Accordingto examples of the presently disclosed subject matter, the IO handler526 searches for the <LU1, LCA2> key in the chunk provisioning datastructure 522, and finds it. This indicates that there is an existingchunk-version data structure 124 for this snapshot layer chunk. The IOhandler 526 is then configured to read the timestamp in the lastchunk-version data structure in the chunk-version data structure 124. Atthis point, there chunk-version data structure 124 only includes asingle chunk-version data element, the one that is associated with thetimestamp T_(—)2, and thus, the IO handler 526 determines that thetimestamp of the last chunk-version data element in the relevantchunk-version data structure 124 is T_(—)2.

Next, the IO handler 526 is configured to query the volume-version datastructure 122 to determine if there was a snapshot operation where thevolume referenced in the write command is a source volume, and if thereis no snapshot operation whose source is the volume referenced in thewrite command, or if all snapshot operations whose source is the volumereferenced in the write command have a timestamp that is earlier than orequal to the timestamp in the last chunk-version data element, the IOhandler 526 can determine that the last chunk-version data element inthe chunk-version data structure can be overwritten. It would beappreciated that there may be multiple snapshot operations which traceback to the volume referenced in the write command and which have atimestamp that is earlier than or equal to the timestamp of the lastchunk-version element.

In the case of the write command that is processed at instant 704 thereis no snapshot operation whose source is the volume referenced in thewrite command, and thus the IO handler 526 is configured to overwritethe last chunk-version data element in the relevant chunk-version datastructure 124 with a chunk-version data element for the current writecommand. According to examples of the presently disclosed subjectmatter, overwriting the last chunk-version data element with achunk-version data element for the current write command can include:overwriting the timestamp listed in the chunk-version data element withthe timestamp for the current write command, and overwriting and/orupdating the storage layer resources that are allocated to thischunk-version data element with the write data from the new writecommand. According to examples of the presently disclosed subjectmatter, the updating of the timestamp and the overwriting/updating ofdata in the storage layer can be carried out as a single atomicoperation.

In the scenario shown in FIG. 7, at time instant 704, the lastchunk-version data element in the chunk-version data structure that isassociated with <LU1, LCA2> is updated, and the timestamp T_(—)2 isreplaced with the timestamp T_(—)3. In addition, the data in the SRAs512-767 is overwritten and/or updated to include the new write data.Further by way of example, if the write data in the write command thatis associated with T_(—)3 overlaps with some (or all) of the write datathat was previously stored in the storage resources that are allocatedto the chunk-version data element, the new write data will overwrite theexisting data. Any non-overlapping data is used to update the allocatedstorage resources (there is no data there old data to overwrite).

Moving forward with the description of FIG. 7, at time instant 706 thechunk-version data structure that is associated with <LU1, LCA2>includes two chunk-version data elements. The chunk-version data elementfrom instant 704 (the one which includes the timestamp T_(—)3 andreferences SRAs 512-767) is retained, and a chunk-version data elementwhich includes the timestamp T_(—)8 and references SRAs 768-1023 isadditionally included in the chunk-version data structure. The inclusionof two (or three, four, . . . , n) chunk-version data elements in asingle chunk-version data structure indicates that, in addition to thelast version of the respective snapshot-layer chunk, a (one or two,three, . . . , (n−1)) previous version(s) of this snapshot-layer chunkis required by the snapshot layer and cannot be overwritten ordiscarded. According to examples of the presently disclosed subjectmatter, the IO handler 126 can be configured to maintain a chunk-versiondata element that is associated with a previous version of therespective snapshot-layer chunk when this respective version of thesnapshot-layer chunk is required by a snapshot volume (or by snapshotvolumes) that was created subsequent to the creation of this version ofthe snapshot-layer chunk. The snapshot volume can be a direct snapshotof the volume with which the chunk-version data element is associated orit can or a source volume of a snapshot volume that is the source volumein a later snapshot operation, in case of a series of snapshotoperations which trace back to the volume referenced in the writecommand.

For illustration, there is provided a description of a series of eventswhich according to examples of the presently disclosed subject mattercould have led to the state of the chunk-version data structure that isassociated with <LU1, LCA2> at time instant 706. The events start aftertime instant 704, when a further write command that is associated withthe snapshot-layer chunk <LU1, LCA2> is received. The write command isreceived when the logical counter is at T_(—)8. According to examples ofthe presently disclosed subject matter, the IO handler 526 searches forthe <LU1, LCA2> key in the chunk provisioning data structure 522, andfinds it. This indicates that there is an existing chunk-version datastructure 124 for this snapshot layer chunk. The IO handler 526 is thenconfigured to read the timestamp in the last chunk-version datastructure in the chunk-version data structure 124. At this point, therechunk-version data structure 124 includes a single chunk-version dataelement which includes the timestamp T_(—)3, and thus, the IO handler526 determines that the timestamp of the last chunk-version data elementin the relevant chunk-version data structure 124 is T_(—)3.

Next, the IO handler 526 is configured to query the volume-version datastructure 122 to determine if there was a snapshot operation where thevolume referenced in the write command is a source volume, and if therewas such a snapshot operation, the IO handler is configured to determinewhether the timestamp that is associated any of such snapshot operationsis earlier than or equal to the timestamp of the last chunk-versionelement in the respective chunk-version data structure. It would beappreciated that there may be multiple snapshot operations which traceback to the volume referenced in the write command and which have atimestamp that is earlier than or equal to the timestamp of the lastchunk-version element.

Thus, according to examples of the presently disclosed subject matter,at time instant 706, the IO handler 526 determines that an overwrite ofthe existing chunk-version data element (the one associated with T_(—)3)is not appropriate (the snapshot from LU1 to LU2 has a timestamp T_(—)7which is later than T_(—)3), and the IO handler 526 creates a newchunk-version data element. For example, the IO handler 526 can requestan allocation of storage layer resources from the free storage layerresources list 524. By way of example, for the chunk-version dataelement that is associated with timestamp T_(—)8, the free storage layerresources list allocated blocks 768-1023. The IO handler updates thechunk provisioning data structure 522, and as a result, at instant 706the chunk-version data structure includes, in addition to thechunk-version data element associated with the timestamp T_(—)3, achunk-version data element that is associated with timestamp T_(—)8, andeach of the two chunk-version data elements is associated with adifferent set of storage layer blocks in a size which corresponds to asnapshot layer chunk.

The state of the chunk version data structure at instant 708 in thescenario illustrated in FIG. 7, by way of example, reflects the receiptof a write command referencing LU2 and LBA 524-526. For example, instant708 immediately follows instant 706. The write command is received whenthe logical counter is at T_(—)8. According to examples of the presentlydisclosed subject matter, the IO handler 526 searches for the <LU2,LCA2> key in the chunk provisioning data structure 522, and finds it. Itis assumed here, that the snapshot layer chunk associated with <LU2,LCA2> had a previous version, but there are no snapshots whose timestampis earlier than or equal to T_(—)8. Accordingly, the write command thatwas received at instant 708 can overwrite or update the previous data,and the same storage layer resource that were allocated to the previousversion of the snapshot layer chunk associated with <LU2, LCA2> can beused for storing the write data from the new write request, as wasdescribed above. The data in the chunk version data element that isassociated with the key <LU2, LCA2> is updated in accordance with thecurrent timestamp and the storage layer resources associated with thewrite operation. As is shown in FIG. 7, the update to chunk version dataelement that is associated with the key <LU2, LCA2> does not affect thechunk version data element that is associated with the key <LU1, LCA2>.

According to a further aspect of the presently disclosed subject matter,there is provided a method of handling reads in a storage system.According to examples of the presently disclosed subject matter, themethod of handling reads in a storage system can include: maintaining avolume-version data structure including volume entries for the LU volumeand for snapshot volumes associated with the LU volume, wherein for eachvolume a respective volume ID is kept, and for each snapshot a timestampfor the snapshot operation and an indication of a connection between arespective target volume and a source volume are kept; maintainingchunk-version data structures, each of which is associated with a volumeID and a logical chunk address which is associated with a plurality oflogical blocks, and each of which includes one or more chunk versiondata elements, wherein each chunk version data element includes atimestamp that indicates the state of a counter when a correspondingwrite to the respective logical chunk address was received and a pointerto storage resources associated with the chunk-version element; inresponse to receiving a read command at the storage system referencing aplurality of logical blocks, initializing a read pointer with a volumeID and a logical chunk address with which the read command isassociated, and setting a time-marker to a current process counterstate; searching for a chunk-version data structure which is associatedwith the chunk read pointer, and if exists, reading data from storageresources referenced in a last chunk-version data element of thechunk-version data structure whose timestamp is not later than a currenttime-marker; wherein in case there are one or more logical blocks thatare referenced in the read command and for which valid data was not yetsearched, searching the volume-version data structure for an indicationof a snapshot operation in which the volume ID in the read pointer is atarget of a snapshot operation, and if the indication exists updatingthe read pointer with the volume ID of a source of the snapshotoperation, and updating the time-marker with the timestamp of thesnapshot operation; and repeating, using the updated read pointer andthe updated time-marker, the search for a chunk-version data structure,the search for a last chunk version data element, the reading of thedata, the search of the volume-version data structure, the update of theread pointer, and the update of the time-marker with the timestamp forthe snapshot operation.

According to examples of the presently disclosed subject matter, theoperations are repeated using an updated read pointer and an updatedtime-marker until all the data associated with one or more of theplurality of logical blocks referenced in the read command is obtained,or if at one of the iterations of the process no chunk version dataelement was found in the chunk-version data structure which isassociated with the (current) chunk read pointer or if there is no chunkversion data element in the chunk-version data structure whose timestampis not later than a current time-marker, adding invalid values to a readresponse for the remaining logical blocks.

For sake of clarity, it should be noted that FIGS. 1-7 and thedescription thereof which was provided above, should be regarded,according to examples of the presently disclosed subject matter, as partof the description of the method of handling reads in a storage system,which is now described. It should be further noted that the method ofhandling reads in a storage system can be implemented in a storagesystem that is designed and managed according to the disclosure providedhereinabove, including with reference to FIGS. 1-7. In further examplesof the presently disclosed subject matter, the system for handling readsin a storage system is not limited by some or all of the teachingsprovided above, in particular with reference to FIGS. 1-7.

Reference is now made to FIG. 8, which is a flowchart illustration of amethod of handling reads in a storage system, according to examples ofthe presently disclosed subject matter. Further by way of example, themethod laid out in FIG. 8 can be implemented by the storage system shownin FIG. 1 and described above. Still further by way of example, themethod laid out in FIG. 8 can be implemented by the storage system shownin FIG. 5 and described above. Accordingly, by way of non-limitingexample, the description of FIG. 8 is made with additional reference tothe storage system shown in FIG. 1 or with reference to the storagesystem shown in FIG. 5.

According to examples of the presently disclosed subject matter, avolume-version data structure including volume entries for a LU volumeand for snapshot volumes associated with the LU volume can be kept(block 805). For each volume a respective volume ID can be kept in thevolume-version data structure, and for each snapshot a timestamp for thesnapshot operation and an indication of a connection between arespective target volume and a source volume can be kept in thevolume-version data structure. The structure and the management of thevolume-version data structure was described above with reference toFIGS. 1-7 and the teachings provided above are also applicable here.

One or more chunk-version data structures can also be maintained, whereeach one of the chunk-version data structures is associated with avolume ID and a logical chunk address (block 810). Each logical chunkaddress is associated with a plurality of logical blocks. In eachchunk-version data structure, one or more chunk version data elementscan be included, where each chunk version data element includes atimestamp at a corresponding write to the respective logical chunkaddress and a pointer to storage resources associated with thechunk-version data element (block 815). The structure and the managementof a chunk-version data structure and of a chunk version data elementwas described above with reference to FIGS. 1-7 and the teachingsprovided above are also applicable here.

At some point, a read command can be received (block 820) at the storagesystem and can be processed in the snapshot layer. The read command canreference a plurality of logical blocks (e.g., LBAs). According toexamples of the presently disclosed subject matter, in response toreceiving the read command at the storage system, a time-marker can beset to a current counter state (block 825). Further by way of example,and still further in response to receiving the read command at thestorage system, a read pointer can be initialized with a volume ID and alogical chunk address with which the read command is associated (block830).

Once the initialization is completed, a iterative process can commence.The iterative process can consist of zero or more (zero, one, two, . . ., n) iterations. At block 835 it is determined whether there is achunk-version data structure which is associated with a current chunkread pointer.

According to examples of the presently disclosed subject matter, if atblock 835 it is determined that there is a chunk-version data structurewhich is associated with the chunk read pointer, the process can processto block 840, where it can be determined whether there is a last chunkversion data element in the chunk version data structure whose timestampis not later than a current time-marker (block 840). If it is determinedthat there is a chunk-version data structure which is associated withthe chunk read pointer, and that there is a chunk version data elementin the chunk version data structure whose timestamp is earlier than orequal to a current time-marker, then the last chunk version data elementin the chunk version data structure whose timestamp is earlier than orequal to a current time-marker is selected. From this last chunk versiondata element, the RSAs which are referenced therein are obtained (e.g.,read), and the data that is stored in the RSAs which are referenced inthe last chunk version data element is read (block 845).

According to examples of the presently disclosed subject matter, thedata that is read in block 845 can be temporarily stored in a readbuffer (not shown). It would be appreciated that several read operationsmay be required in order to generate a complete response to the readcommand.

After the data is read from the storage layer, in block 850 it isdetermined the process was implemented for all of the LBAs that werereferenced in the read command. In case in block 850 it is determinedthat the process was indeed implemented for all of the LBAs that werereferenced in the read command, a response to the read command can begenerated with the data that was obtained from the storage layer and/orwith invalid data indication(s) for LBAs that were referenced in theread command and for which valid data could not be obtained from thestorage system (block 855). The invalid indications can be, for example,zero values.

Resuming the description of the examples of the presently disclosedsubject matter illustrated in FIG. 8, if at block 835 it is determinedthat there is no chunk-version data structure which is associated with acurrent chunk read pointer, or if is determined, after the execution ofblock 850, that a further iteration of the process may be required,i.e., when it is determined at block 850 that there is at least one moreLBA that was referenced in the read command and in respect of which thesearch for write data in the storage layer was not yet implemented orcompleted, block 860 may be implemented, where it can be determinedwhether there is a snapshot operation in which the volume referenced inthe read pointer is a target of a snapshot operation whose associatedtimestamp is earlier than or equal to the current time-marker.

If it is determined, e.g., according to the volume version datastructure, that there exists a snapshot operation in which the volumereferenced in the read pointer is a target of a snapshot operation, andthat snapshot operation is associated with a timestamp that is earlierthan or equal to the current time-marker, the read pointer can beupdated with the volume ID of the snapshot source (block 865), and thetime-marker can be updated with the timestamp immediately preceding thetimestamp of the snapshot operation (snapshot timestamp—1) (block 870),and from there the process returns to block 835, which was describedabove.

According to examples of the presently disclosed subject matter, theevaluation in block 860 can be implemented when:

-   -   a. after the execution of block 835 it is determined there is no        chunk-version data structure which is associated with the        current chunk read pointer, or    -   b. after the execution of block 840 it is determined that in the        chunk-version data structure which is associated with the        current chunk read pointer there isn't a chunk version data        element whose timestamp is earlier than or equal to a current        time-marker, or    -   c. after the execution of block 850 it is determined that a        further iteration of the process may be required, it is        determined at block 850 that there is at least one more LBA that        was referenced in the read command and in respect of which the        search for write data in the storage layer was not yet        implemented or completed.

According to further examples of the presently disclosed subject matter,a response to the read command can be generated with the data that wasobtained from the storage layer and/or with invalid data indication(s)for LBAs that were referenced in the read command and for which validdata could not be obtained from the storage system (block 855), when itis determined that:

-   -   a. after the execution of block 850 it is determined that the        search for write data in the storage layer was carried out for        each one of the LBAs referenced in the write command, or    -   b. after the execution of block 860 it is determined that there        is no snapshot operation in which the volume referenced in the        read pointer is a target of a snapshot operation whose        associated timestamp is earlier than or equal to the current        time-marker.

It would be appreciated that in some examples of the presently disclosedsubject matter, the process illustrated in FIG. 8 when implemented withthe volume version data structure, the chunk version data structure andpossibly also the chunk provisioning data structure and the free storagelayer resources list, can support an efficient and reliable storagesystem which implements snapshots (including revert operations) andmanages multiple chunk layer versions.

There is provided below Table 1 that is accompanied by FIG. 9, whichtogether illustrate an example of a scenario that involves a series ofoperations that are carried out according to examples of the presentlydisclosed subject matter. The operations in Table 1 are responsive tosnapshot create commands, volume restore command, read commands andwrite commands, and are processed using examples of a method of managinga storage system, examples of a method of handling writes in a storagesystem, and examples of a method of handling reads in a storage system,all of which according to examples of the presently disclosed subjectmatter. FIG. 9 provides a schematic illustration of one example of apossible implementation of the volume-version data structure andcorresponds to operations included in Table 1, according to examples ofthe presently disclosed subject matter.

Each row in Table 1 is associated with a certain snapshot createcommand, volume restore command, read command or write command. Thecells in the leftmost column in Table 1 list the logical timestamp thatis associated with the IO command with which the respective row isassociated. The cells in the second to last column from the left, eachinclude a description of the IO command to which the respective rowrelates. The cells in the second to last column from the right, providea description of the main operations that take place in the snapshotlayer in response to the respective IO command. The cells in therightmost column provide the number of the storage layer chunk that isinvolved in the servicing of the IO command. In this regard, it would berecalled that the free storage layer resources list 524 that wasdescribed above with reference to FIG. 5 can hold storage layer chunkswhich are enumerated, with each storage layer chunk being associatedwith a corresponding set of storage layer blocks of a certain size.

TABLE 1 Logical Storage Layer Timestamp IO Command Snapshot IO algorithmChunk No. T_0 User creates A volume-version data structure is created LU1* and LU1 is added to the volume-version data structure (at the root).T_0 Write to Is there a snapshot layer chunk registered in 0000 LU1*,LBA the chunk provisioning data structure that is 1000 associated with<LU1, LBA1000> → No Request an allocation of storage layer resourcesfrom the free storage layer resource list → storage layer chunk <0000>is allocated Indicate in the chunk provisioning data structure that thesnapshot layer chunk that is associated with <LU1, LBA1000> wasprovisioned, and create a chunk-version data structure and register itin the chunk provisioning data structure using the key <LU1, LBA1000>.Create a chunk-version data element in the chunk-version data structurethat is associated with <LU1, LBA1000> and include in the chunk-versiondata element the timestamp <T_0> and the allocated storage layer chunk<0000>. T_0 Read from Is there a chunk version data structure that is0000 LU1*, LBA associated with <LU1, LBA1000>? → YES 1000 Is there achunk version data element (in the chunk version data structure that isassociated with <LU1, LBA1000>) whose timestamp is earlier than or equalto T_0? → YES Select the last chunk version data element (in the chunkversion data structure that is associated with <LU1, LBA1000>) whosetimestamp is earlier than or eual to T_0 and read data from the storagelayer according to the storage layer chunk no. referenced in theselected chunk version data element. → Read data from 0000 Searchconducted for all logical blocks referenced in the read command? → YESRespond to read command → Generate read response with the data stored inthe SRAs associated with storage layer chunk no. 0000 T_0 Read to LU1,Is there a chunk version data structure that is Pointer LBA 3000associated with <LU1, LBA3000>? → No to zero Respond to read command →Generate read buffer response with invalid values (zeros) to indicateformatted data in the storage system for the logical address referencedin the write command. T_1 Create Add LU2 to the volume-version dataSnapshot structure with a connector from LU1 (the from LU 1* sourcevolume) to LU2 (the target volume) to to LU 2 indicate a snapshotoperation and from LU1 to LU2 and on the connector write the timestamp<T_1> T_1 Read from Is there a chunk version data structure that is 0000LU2, LBA associated with <LU2, LBA1000>? → No 1000 Is there a snapshotoperation in which LU2 is a target and which is associated with atimestamp that is earlier than or equal to 1? → Yes (LU1) Update theread pointer with the volume ID of the snapshot source → read pointer =LU1 Update the time marker with the timestamp for the snapshot operation−1 →time marker = T_0 Is there a chunk version data structure that isassociated with <LU1, LBA1000>? → Yes Is there a chunk version dataelement (in the chunk version data structure that is associated with<LU1, LBA1000>) whose timestamp is earlier than or equal to T_0? → YESSelect the last chunk version data element (in the chunk version datastructure that is associated with <LU1, LBA1000>) whose timestamp isearlier than or equal to T_0 and read data from the storage layeraccording to the storage layer chunk no. referenced in the selectedchunk version data element. → READ DATA FROM 0000 Search conducted forall logical blocks referenced in the read command? → YES Respond to readcommand → Generate read response with the data stored in the SRAsassociated with storage layer chunk no. 0000 T_2 Create Add LU3 to thevolume-version data Snapshot structure with a connector from LU2 (thefrom LU 2 to source volume) to LU3 (the target volume) to LU 3 indicatea snapshot operation and from LU2 to LU3 and on the connector write thetimestamp <T_2> T_3 Create Add LU4 to the volume-version data Snapshotstructure with a connector from LU1 (the from LU 1* source volume) toLU4 (the target volume) to to LU 4 indicate a snapshot operation andfrom LU1 to LU4 and on the connector write the timestamp <T_3> T_4Create Add LU5 to the volume-version data Snapshot structure with aconnector from LU2 (the from LU 2 to source volume) to LU5 (the targetvolume) to LU 5 indicate a snapshot operation and from LU2 to LU5 and onthe connector write the timestamp <T_4> T_4 Write to LU1, Is there asnapshot layer chunk registered in 1000 LBA 3000 the chunk provisioningdata structure that is associated with <LU1, LBA3000> → No Request anallocation of storage layer resources from the free storage layerresource list → Storage layer chunk <1000> is allocated Indicate in thechunk provisioning data structure that the snapshot layer chunk that isassociated with <LU1, LBA3000> was provisioned, and create achunk-version data structure and register it in the chunk provisioningdata structure using the key <LU1, LBA3000>. Create a chunk-version dataelement in the chunk-version data structure that is associated with<LU1, LBA3000> and include in the chunk-version data element thetimestamp <T_4> and the allocated storage layer chunk <1000>. T_4 Writeto LU5, Is there a snapshot layer chunk registered in 2000 LBA 3000 thechunk provisioning data structure that is associated with <LU5, LBA3000>→ No Request an allocation of storage layer resources from the freestorage layer resource list → Storage layer chunk <2000> is allocatedIndicate in the chunk provisioning data structure that the snapshotlayer chunk that is associated with <LU5, LBA3000> was provisioned, andcreate a chunk-version data structure and register it in the chunkprovisioning data structure using the key <LU5, LBA3000>. Create achunk-version data element in the chunk-version data structure that isassociated with <LU5, LBA3000> and include in the chunk-version dataelement the timestamp <T_4> and the allocated storage layer chunk<2000>. T_4 Read from Is there a chunk version data structure that is1000 LU1, LBA associated with <LU1, LBA3000>? → YES 3000 Is there achunk version data element (in the chunk version data structure that isassociated with <LU1, LBA3000>) whose timestamp is no greater than T_4?→ YES Select the last chunk version data element (in the chunk versiondata structure that is associated with <LU1, LBA3000>) whose timestampis no greater than (earlier than or equal to) T_4 and read data from thestorage layer according to the storage layer chunk no. referenced in theselected chunk version data element. → READ DATA FROM 1000 Searchconducted for all logical blocks referenced in the read command? → YESRespond to read command → Generate read response with the data stored inthe sras associated with storage layer chunk no. 1000 T_5 Create Add anew node to the volume-version data Snapshot of structure referencedwith LU1 and add a type connector from LU5 (the source volume) to‘REVERT’ LU1 (the target volume) to indicate a from LU 5 to snapshotoperation and from LU5 to LU1, on LU 1 the connector write the timestamp<T_5> mark the previous node that was reference with LU1 as ‘REMOVED’T_5 Read from Is there a chunk version data structure that is 2000 LU1,LBA associated with <LU1, LBA3000>? → No 3000 Is there a snapshotoperation in which LU1 is a target and which is associated with atimestamp that is earlier than or equal to 5? → Yes (LU5) Update theread pointer with the volume ID of the snapshot source → read pointer =LU5 Update the time marker with the timestamp for the snapshot operation−1 →time marker = T_4 Is there a chunk version data structure that isassociated with <LU5, LBA3000>? → Yes Is there a chunk version dataelement (in the chunk version data structure that is associated with<LU5, LBA3000>) whose timestamp is earlier than or equal to T_4? → YESSelect the last chunk version data element (in the chunk version datastructure that is associated with <LU5, LBA3000>) whose timestamp is nogreater than T_4 and read data from the storage layer according to thestorage layer chunk no. referenced in the selected chunk version dataelement. → Read data from 2000 Search conducted for all logical blocksreferenced in the read command? → YES Respond to read command → Generateread response with the data stored in the SRAs associated with storagelayer chunk no. 2000 T_6 Create Add LU6 to the volume-version dataSnapshot structure with a connector from LU1 (the from LU 1 to sourcevolume) to LU6 (the target volume) to LU 6 indicate a snapshot operationand from LU1 to LU6 and on the connector write the timestamp <T_6> T_6Write to LU2, Is there a snapshot layer chunk registered in 3000 LBA1000 the chunk provisioning data structure that is associated with <LU2,LBA1000> → No Request an allocation of storage layer resources from thefree storage layer resource list → Storage layer chunk <3000> isallocated Indicate in the chunk provisioning data structure that thesnapshot layer chunk that is associated with <LU2, LBA1000> wasprovisioned, and create a chunk-version data structure and register itin the chunk provisioning data structure using the key <LU2, LBA1000>.Create a chunk-version data element in the chunk-version data structurethat is associated with <LU2, LBA1000> and include in the chunk-versiondata element the timestamp <T_6> and the allocated storage layer chunk<3000>. T_7 Create Add LU7 to the volume-version data Snapshot structurewith a connector from LU6 (the from LU 6 to source volume) to LU7 (thetarget volume) to LU 7 indicate a snapshot operation and from LU6 to LU7and on the connector write the timestamp <T_7> T_7 Write to LU1, Isthere a snapshot layer chunk registered in 4000 LBA 1000 the chunkprovisioning data structure that is associated with <LU1, LBA1000> → NoRequest an allocation of storage layer resources from the free storagelayer resource list → Storage layer chunk <4000> is allocated Indicatein the chunk provisioning data structure that the snapshot layer chunkthat is associated with <LU1, LBA1000> was provisioned, and create achunk-version data structure and register it in the chunk provisioningdata structure using the key <LU1, LBA1000>. Create a chunk-version dataelement in the chunk-version data structure that is associated with<LU1, LBA1000> and include in the chunk-version data element thetimestamp <T_7> and the allocated storage layer chunk <4000>. T_7 Writeto LU2, Is there a snapshot layer chunk registered in 3000 LBA 1000 thechunk provisioning data structure that is associated with <LU2, LBA1000>→ YES → Read a timestamp of a last chunk-version element in the existingchunk-version data structure → T_6. Is there a snapshot operation whosesource is the volume referenced in the write command (LU_2)?→ YES at T_2and at T_4 → are all snapshot operations whose source is LU_2 have atimestamp that is earlier than or equal to the timestamp of the lastchunk- version data element in the chunk version data structure <LU2,LBA1000>? → YES (the timestamp of the last chunk-version data element inthe chunk version data structure <LU2, LBA1000> is T_6). Overwrite thelast chunk-version data element in the chunk version data structure thatis associated with <LU2, LBA1000> with a chunk version data element thatincludes a reference to storage layer chunk <3000> and to timestamp T_7.T_8 Create Add LU8 to the volume-version data Snapshot structure with aconnector from LU6 (the from LU6 to source volume) to LU8 (the targetvolume) to LU8 indicate a snapshot operation and from LU6 to LU8 and onthe connector write the timestamp <T_8>

It will also be understood that the system according to the inventionmay be a suitably programmed computer. Likewise, the inventioncontemplates a computer program being readable by a computer forexecuting the method of the invention. The invention furthercontemplates a machine-readable memory tangibly embodying a program ofinstructions executable by the machine for executing the method of theinvention.

1. A method of managing data in a storage system, comprising:maintaining a volume-version data structure including LU volume entriesfor a LU volume and for snapshot volumes associated with the LU volume,wherein for each volume a respective volume ID is kept, and for eachsnapshot a timestamp for the snapshot operation and an indication of aconnection between a respective target volume and a source volume arekept; maintaining a chunk-version data structure that is uniquelyassociated with a volume ID and a logical chunk address combination;including in the chunk-version data structure one or more chunk-versiondata elements, each chunk version data element including a timestamp ata corresponding write to the respective logical chunk address in therespective volume and a pointer to storage resources associated with thechunk-version data element.
 2. The method according to claim 1, whereinresponsive to an instruction to create a new snapshot volume where asource of the snapshot is the LU volume or a snapshot volume that isassociated with the LU volume, updating the volume-version datastructure with: a volume ID reference for the new snapshot-volume, anindication of a connection between the new snapshot-volume and thesource, and a timestamp for the snapshot operation.
 3. The methodaccording to claim 1, further comprising responsive to receiving a writecommand in the storage system: determining with which snapshot layerchunk the write command is associated; and determining whether there isan existing chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated.
 4. Themethod according to claim 3, wherein in case there is no existing chunkversion data structure which is associated with the snapshot layer chunkwith which the write command is associated, generating a new chunkversion data structure that is uniquely associated with the volume IDand with a logical chunk address of the snapshot layer chunk with whichthe write command is associated, and generating in the chunk-versiondata structure a chunk-version data element including a currenttimestamp and a pointer to storage resources that were allocated forthis chunk-version data element.
 5. The method according to claim 3,wherein in case there is an existing chunk version data structure whichis associated with the snapshot layer chunk with which the write commandis associated, determining whether a new chunk version data elementneeds to be generated or whether an existing chunk version data elementcan be overwritten.
 6. The method according to claim 5, wherein in caseit is determined that a new chunk version data element needs to begenerated, allocating storage resources for the new chunk version dataelement.
 7. The method according to claim 6, further comprisingrequesting allocation of storage resources for the new chunk versiondata element from a free storage layer resources list.
 8. The methodaccording to claim 5, wherein in case it is determined that there is anexisting chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated, andthat an existing chunk version data element can be overwritten, writingwrite data of the write command in storage resources that were allocatedto a last chunk version data element in the chunk version datastructure.
 9. The method according to claim 8, wherein in response towriting the write data of the write command in the storage resourcesthat were allocated to the last chunk version data element in the chunkversion data structure updating the timestamp of the last chunk versiondata element with a current timestamp.
 10. The method according to claim5, wherein determining whether a new chunk version data element needs tobe generated or whether an existing chunk version data element can beoverwritten, comprises: reading a timestamp from a last chunk-versiondata element in the existing chunk-version data structure; and if thereis no snapshot operation whose source is the volume referenced in thewrite command, or if all snapshot operations whose source is the volumereferenced in the write command have a timestamp that is earlier than orequal to the timestamp in the last chunk-version data element,determining that the last chunk-version data element in the existingchunk-version data structure can be overwritten.
 11. A storage system,comprising: a storage layer providing storage resources; a snapshotlayer comprising: a volume-version data structure including LU volumeentries for an LU volume and for snapshot volumes associated with the LUvolume, wherein for each volume a respective volume ID is kept, and foreach snapshot a timestamp for the snapshot operation and an indicationof a connection between a respective target volume and a source volumeare kept; a chunk-version data structure that is uniquely associatedwith a volume ID and a logical chunk address combination; a IO handlerthat is configured to include in the chunk-version data structure one ormore chunk-version data elements, each chunk version data elementincluding a timestamp at a corresponding write to the respective logicalchunk address in the respective volume and a pointer to storageresources associated with the chunk-version data element.
 12. Thestorage system according to claim 11, wherein responsive to aninstruction to create a new snapshot volume where a source of thesnapshot is the LU volume or a snapshot volume that is associated withthe LU volume, the IO handler is configured to update the volume-versiondata structure with: a volume ID reference for the new snapshot-volume,an indication of a connection between the new snapshot-volume and thesource, and a timestamp for the snapshot operation.
 13. The storagesystem according to claim 11, further comprising responsive to receivinga write command in the storage system: the IO handler is configured todetermine with which snapshot layer chunk the write command isassociated; and the IO handler is configured to determine whether thereis an existing chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated.
 14. Thestorage system according to claim 13, wherein in case there is noexisting chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated, the IOhandler is configured to generate a new chunk version data structurethat is uniquely associated with the volume ID and with a logical chunkaddress of the snapshot layer chunk with which the write command isassociated, and to generate in the chunk-version data structure achunk-version data element including a current timestamp and a pointerto storage resources that were allocated for this chunk-version dataelement.
 15. The storage system according to claim 13, wherein in casethere is an existing chunk version data structure which is associatedwith the snapshot layer chunk with which the write command isassociated, the IO handler is configured to determine whether a newchunk version data element needs to be generated or whether an existingchunk version data element can be overwritten.
 16. The storage systemaccording to claim 15, further comprising a free storage resources list,and wherein in case it is determined that a new chunk version dataelement needs to be generated, the IO handler is configured to requestfrom the free storage resources list allocation of storage resources forthe new chunk version data element.
 17. The storage system according toclaim 14, wherein the IO handler is responsive to determining there isan existing chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated, andthat an existing chunk version data element can be overwritten, forwriting write data of the write command in storage resources that wereallocated to last chunk version data element in the chunk version datastructure and for updating the timestamp of the last chunk version dataelement with a current timestamp.
 18. The storage system according toclaim 14, wherein the IO handler is configured to determine whether anew chunk version data element needs to be generated or whether anexisting chunk version data element can be overwritten, by: reading atimestamp from a last chunk-version data element in the existingchunk-version data structure; and if there is no snapshot operationwhose source is the volume referenced in the write command, or if allsnapshot operations whose source is the volume referenced in the writecommand have a timestamp that is earlier than or equal to the timestampin the last chunk-version data element, the IO handler is configured todetermine that the last chunk-version data element in the existingchunk-version data structure can be overwritten.
 19. A program storagedevice readable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform a method of managing data in astorage system, comprising: maintaining a volume-version data structureincluding LU volume entries for an LU volume and for snapshot volumesassociated with the LU volume, wherein for each volume a respectivevolume ID is kept, and for each snapshot a timestamp for the snapshotoperation and an indication of a connection between a respective targetvolume and a source volume are kept; maintaining a chunk-version datastructure that is uniquely associated with a volume ID and a logicalchunk address combination; and including in the chunk-version datastructure one or more chunk-version data elements, each chunk versiondata element including a timestamp at a corresponding write to therespective logical chunk address in the respective volume and a pointerto storage resources associated with the chunk-version data element. 20.A computer program product comprising a computer useable medium havingcomputer readable program code embodied therein for managing data in astorage system, the computer program product comprising: computerreadable program code for causing the computer to maintain avolume-version data structure including LU volume entries for an LUvolume and for snapshot volumes associated with the LU volume, whereinfor each volume a respective volume ID is kept, and for each snapshot atimestamp for the snapshot operation and an indication of a connectionbetween a respective target volume and a source volume are kept;computer readable program code for causing the computer to maintain achunk-version data structure that is uniquely associated with a volumeID and a logical chunk address combination; and computer readableprogram code for causing the computer to include in the chunk-versiondata structure one or more chunk-version data elements, each chunkversion data element including a timestamp at a corresponding write tothe respective logical chunk address in the respective volume and apointer to storage resources associated with the chunk-version dataelement.
 21. A method of handling writes in a storage system,comprising: in response to receiving a write command determining withwhich snapshot layer chunk the write command is associated, and whetherthere is an existing chunk version data structure which is associatedwith the snapshot layer chunk with which the write command isassociated; in case the write command is associated with an existingchunk-version data structure, reading a timestamp from a lastchunk-version data element in the existing chunk-version data structure,and if there is a snapshot operation whose source is a volume referencedin the write command and this snapshot operation is associated with atimestamp that is later than a timestamp of the last chunk-version dataelement, maintaining the last chunk-version data element and adding tothe existing chunk-version data structure a new chunk version dataelement for the write command.
 22. The method according to claim 21,wherein adding to the existing chunk-version data structure a new chunkversion data element includes adding a reference to storage resourcesassociated with the new chunk-version data element.
 23. The methodaccording to claim 22, further comprising requesting allocation ofstorage resources for the new chunk version data element from a freestorage layer resources list.
 24. The method according to claim 21,further comprising maintaining a volume-version data structure includingvolume entries for a LU volume and for snapshot volumes associated withthe LU volume, wherein for each volume a respective volume ID is kept,and for each snapshot a timestamp for the snapshot operation and anindication of a connection between a respective target volume and asource volume are kept.
 25. The method according to claim 24, furthercomprising maintaining chunk-version data structures, each of which isassociated with a volume ID and a logical chunk address, and each ofwhich includes one or more chunk version data elements, wherein eachchunk version data element includes a timestamp at a corresponding writeto the respective logical chunk address and a pointer to storageresources associated with the chunk-version data element.
 26. The methodaccording to claim 25, wherein in case there is no existing chunkversion data structure which is associated with the snapshot layer chunkwith which the write command is associated, generating a new chunkversion data structure that is uniquely associated with the volume IDand with a logical chunk address of the snapshot layer chunk with whichthe write command is associated, and generating in the chunk-versiondata structure a new chunk-version data element including a timestamp atthe write command and a pointer to storage resources that were allocatedfor this chunk-version data element.
 27. The method according to claim25, in case there is an existing chunk version data structure which isassociated with the snapshot layer chunk with which the write command isassociated, and there is an existing chunk version data element in theexisting chunk version data structure that can be overwritten, writingwrite data of the write command in storage resources that were allocatedto a last chunk version data element in the existing chunk version datastructure.
 28. The method according to claim 27, comprising determiningthat the existing chunk version data element in the existing chunkversion data structure can be overwritten when the write command isassociated with an existing chunk-version data structure, and thetimestamp of the last chunk-version data element in the existingchunk-version data structure is later than or equal to the timestamp ofa snapshot operation in which the source volume is the volume referencedin the write command.
 29. A storage system, comprising: a storage layerproviding storage resources; a snapshot layer comprising: avolume-version data structure; a chunk-version data structure; a IOhandler responsive to receiving a write command in the snapshot layerfor determining with which snapshot layer chunk the write command isassociated, and whether there is an existing chunk version datastructure which is associated with the snapshot layer chunk with whichthe write command is associated, and in case the write command isassociated with an existing chunk-version data structure, the IO handleris configured to read a timestamp from a last chunk-version data elementin the existing chunk-version data structure, and if there is a snapshotoperation whose source is a volume referenced in the write command andthis snapshot operation is associated with a timestamp that is laterthan a timestamp of the last chunk-version data element, the IO handleris configured to maintain the last chunk-version data element and to addto the existing chunk-version data structure a new chunk version dataelement for the write command.
 30. The storage system according to claim29, wherein the IO handler is configured to including in the new chunkversion data element a reference to storage resources which areassociated with the new chunk-version data element.
 31. The storagesystem according to claim 30, further comprising a free storageresources list, and wherein in case it is determined that a new chunkversion data element needs to be generated, the IO handler is configuredto request from the free storage resources list allocation of storageresources for the new chunk version data element.
 32. The storage systemaccording to claim 29, wherein the volume-version data structureincludes volume entries for a LU volume and for snapshot volumesassociated with the LU volume, wherein for each volume a respectivevolume ID is kept, and for each snapshot a timestamp for the snapshotoperation and an indication of a connection between a respective targetvolume and a source volume are kept.
 33. The storage system according toclaim 32, wherein the chunk-version data structure is associated with avolume ID and a logical chunk address, and includes one or more chunkversion data elements, wherein each chunk version data element includesa timestamp at a corresponding write to the respective logical chunkaddress and a pointer to storage resources associated with thechunk-version data element.
 34. The storage system according to claim33, wherein in case the IO handler determines that there is no existingchunk version data structure which is associated with the snapshot layerchunk with which the write command is associated, the IO handler isconfigured to generate a new chunk version data structure that isuniquely associated with the volume ID and with a logical chunk addressof the snapshot layer chunk with which the write command is associated,and to generate in the chunk-version data structure a new chunk-versiondata element including a timestamp at the write command and a pointer tostorage resources that were allocated for this chunk-version dataelement.
 35. The storage system according to claim 33, in case the IOhander determines that there is an existing chunk version data structurewhich is associated with the snapshot layer chunk with which the writecommand is associated, and that there is an existing chunk version dataelement in the existing chunk version data structure that can beoverwritten, the IO handler is configured to write write-data of thewrite command in storage resources that were allocated to a last chunkversion data element in the existing chunk version data structure. 36.The storage system according to claim 35, wherein the IO handler isconfigured to determine that the existing chunk version data element inthe existing chunk version data structure can be overwritten when thewrite command is associated with an existing chunk-version datastructure, and the timestamp of the last chunk-version data element inthe existing chunk-version data structure is later than or equal to thetimestamp of a snapshot operation in which the source volume is thevolume referenced in the write command.
 37. A program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform a method of handling writes in astorage system, comprising: in response to receiving a write commanddetermining with which snapshot layer chunk the write command isassociated, and whether there is an existing chunk version datastructure which is associated with the snapshot layer chunk with whichthe write command is associated; in case the write command is associatedwith an existing chunk-version data structure, reading a timestamp froma last chunk-version data element in the existing chunk-version datastructure, and if there is a snapshot operation whose source is a volumereferenced in the write command and this snapshot operation isassociated with a timestamp that is later than a timestamp of the lastchunk-version data element, maintaining the last chunk-version dataelement and adding to the existing chunk-version data structure a newchunk version data element for the write command.
 38. A computer programproduct comprising a computer useable medium having computer readableprogram code embodied therein for handling writes in a storage system,the computer program product comprising: computer readable program codethat is responsive to receiving a write command in the storage systemfor causing the computer to determine with which snapshot layer chunkthe write command is associated, and to determine whether there is anexisting chunk version data structure which is associated with thesnapshot layer chunk with which the write command is associated;computer readable program code for causing the computer to determinewhether the write command is associated with an existing chunk-versiondata structure, and in case the write command is associated with anexisting chunk-version data structure, the computer readable programcode is configured to cause a timestamp from a last chunk-version dataelement in the existing chunk-version data structure to be read, and ifthere is a snapshot operation whose source is a volume referenced in thewrite command and this snapshot operation is associated with a timestampthat is later than a timestamp of the last chunk-version data element,the computer readable program code is configured to cause the lastchunk-version data element to maintained and a new chunk version dataelement for the write command to be added to the existing chunk-versiondata structure.
 39. A method of handling reads in a storage systemcomprising: (a) in response to receiving a read command at the storagesystem determining with which volume and with which snapshot layer chunkthe read command is associated, initializing a read pointer withrespective volume ID and logical chunk address, and initializing atime-marker according to a timestamp at receipt of the read command; (b)determining whether there is a chunk-version data structure that isassociated with a current read pointer; (c1) if the condition inoperation (b) is not met, determining whether there is a snapshotoperation where the volume that is referenced in the write pointer is atarget volume and whether the snapshot operation is associated with atimestamp that is not greater than the current time-marker; and (d1) ifthe conditions in operation (c) are met updating the read pointer with avolume ID of a source volume of the snapshot operation, updating thetime-marker with the timestamp for the snapshot operation—1, andreturning to operation (b).
 40. The method according to claim 39,further comprising: (c2) if the condition in operation (b) is met,determining whether in the chunk version data structure there is a chunkversion data element whose timestamp is not greater than a currenttime-marker; and (e1) if the condition in operation (c2) is met readingdata from the storage resource referenced in a last chunk versionelement of the chunk version data structure.
 41. The method according toclaim 40, wherein in response to implementing operation (e1): (f)determining whether a search was conducted for all logical blocksreferenced in the read command; (g1) and if the condition in operation(f) is met responding to the read command with data obtain in operation(e1) and including invalid data indication(s) for data blocks referencedin the read command for which valid data could not be obtained.
 42. Themethod according to claim 40, wherein in case the condition in operation(c2) is not met, returning to operation (c1).
 43. The method accordingto claim 41, wherein in case the condition in operation (f) is not met,returning to operation (c1).
 44. The method according to claim 38,further comprising maintaining a volume-version data structure includingvolume entries for a LU volume and for snapshot volumes associated withthe LU volume, wherein for each volume a respective volume ID is kept,and for each snapshot a timestamp for the snapshot operation and anindication of a connection between a respective target volume and asource volume are kept.
 45. The method according to claim 44, furthercomprising maintaining chunk-version data structures, each of which isassociated with a volume ID and a logical chunk address, and each ofwhich includes one or more chunk version data elements, wherein eachchunk version data element includes a timestamp of a corresponding writeto the respective logical chunk address and a pointer to storageresources associated with the chunk-version data element.
 46. A storagesystem, comprising: a storage layer providing storage resources; asnapshot layer comprising: a volume-version data structure; achunk-version data structure; a IO handler responsive to receiving aread comment for: (a) determining with which volume and with whichsnapshot layer chunk the read command is associated, initializing a readpointer with respective volume ID and logical chunk address, andinitializing a time-marker according to a timestamp of the read command;(b) determining whether there is a chunk-version data structure that isassociated with a current read pointer; (c1) if the condition inoperation (b) is not met, determining whether there is a snapshotoperation where the volume that is referenced in the write pointer is atarget volume and whether the snapshot operation is associated with atimestamp that is not greater than the current time-marker; and (d1) ifthe conditions in operation (c) are met the IO under is configured for:updating the read pointer with a volume ID of a source volume of thesnapshot operation, updating the time-marker with the timestamp for thesnapshot operation—1, and returning to operation (b).
 47. The storagesystem according to claim 46, wherein in case the condition in operation(b) is met the IO handler is configured for: (c2) determining whether inthe chunk version data structure there is a chunk version data elementwhose timestamp is not greater than a current time-marker; and (e1) ifthe condition in operation (c2) is met the IO handler is configured forreading data from the storage resource referenced in a last chunkversion element of the chunk version data structure.
 48. The storagesystem according to claim 47, wherein in response to implementingoperation (e1), the IO hander is configured for: (f) determining whethera search was conducted for all logical blocks referenced in the readcommand; (g1) and if the condition in operation (f) is met, the IOhandler is configured for responding to the read command with dataobtain in operation (e1) and including invalid data indication(s) fordata blocks referenced in the read command for which valid data couldnot be obtained.
 49. The storage system according to claim 47, whereinin case the condition in operation (c2) is not met, the IO handler isconfigured to return to operation (c1).
 50. The method according toclaim 41, wherein in case the condition in operation (f) is not met, theIO handler is configured to return to operation (c1).
 51. The storagesystem according to claim 46, wherein the volume-version data structureincludes volume entries for a LU volume and for snapshot volumesassociated with the LU volume, and wherein for each volume a respectivevolume ID is kept, and for each snapshot a timestamp for the snapshotoperation and an indication of a connection between a respective targetvolume and a source volume are kept.
 52. The method according to claim51, wherein each one of the chunk-version data structures is associatedwith a volume ID and a logical chunk address, and includes one or morechunk version data elements, wherein each chunk version data elementincludes a timestamp of a corresponding write to the respective logicalchunk address and a pointer to storage resources associated with thechunk-version data element.
 53. A program storage device readable bymachine, tangibly embodying a program of instructions executable by themachine to perform a method of handling reads in a storage systemcomprising: (a) in response to receiving a read command at the storagesystem determining with which volume and with which snapshot layer chunkthe read command is associated, initializing a read pointer withrespective volume ID and logical chunk address, and initializing atime-marker according to a timestamp of the read command; (b)determining whether there is a chunk-version data structure that isassociated with a current read pointer; (c1) if the condition inoperation (b) is not met, determining whether there is a snapshotoperation where the volume that is referenced in the write pointer is atarget volume and whether the snapshot operation is associated with atimestamp that is not greater than the current time-marker; and (d1) ifthe conditions in operation (c) are met updating the read pointer with avolume ID of a source volume of the snapshot operation, updating thetime-marker with the timestamp for the snapshot operation—1, andreturning to operation (b).
 54. The method according to claim 1, whereinfurther in response to the instruction to create a new snapshot volumesuspending IO to the source volume and incrementing a timestamp for thesnapshot operation, and once the snapshot operation is complete,enabling IOs to the source volume to resume.
 55. The method according to4, wherein the timestamp is a logical timestamp, and wherein the logicaltimestamp is incremented as a result of a volume create, a snapshotcreate or a revert operation.